Selected features

  • Simple yet Powerful Embedded Test Scripting
    Web Roller has excellent embedded abilities of generating test data dynamically. Request parameters and URI's can be calculated dynamically. Test functionality can be extended through several standard programming languages, including JavaScript, VBScript and PerlScript supported by Windows™ ActiveScriptingHost. Additionally, Web Roller provides you with wide range of test parameters. Most of global test settings can be redefined within individual requests.

  • More Load from Less Hardware
    Web Roller is used for performance testing and provides high performance itself. The application itself is light and fast - it does not consume lots of system resources and provides high performance. Thus it, first, gives the ability to provide a significant load on the tested server/application at the same time allowing to run at very modest hardware requirements, and, second, ensures accuracy of testing results. Web Roller allows you to generate substantial load without purchasing additional testing hardware or losing precision. By producing more load from less hardware, Web Roller makes performance testing effortless and cost-effective.

  • Accurate Real User Simulation
    Web Roller is extremely accurate in simulating real user actions. Each virtual user is absolutely independent from other ones. Each has cookies, data and parameters of its own. Web Roller provides proxy header substitution to emulate activity of multiple users coming from different IP's. Use of scripting gives you almost unlimited capabilities of simulating user behavior.
    Since Web Roller is based on recording real interaction between browser and server, it provides extremely correct emulation of user actions.

  • Create Load Tests Quickly and Easily
    Web Roller offers an integrated testing environment allowing to perform all tasks (from initial design of test sequences to final reporting) without leaving the application.
    Web Roller is simple in use and training. To start simple testing you will need a minimal experience. And, the more you learn about Web Roller, the more of its power you will master.

  • Charts and Reports
    Web Roller provides you with various charts that report you how tests were performed. You are able to generate reports in text (suitable for immediate loading in MS Excel) and HTML formats. Reports can include both raw and summarized data. presenting various information (such as response times vs. amount of virtual users, transaction performance summaries, failed requests etc.).

  • Secure Content Testing
    Web Roller supports SSL, digital server certificates and other security features related to HTTPS.

  • Functional Testing is not a Problem
    Web Roller is created for performance, load and stress testing. Yet it is suitable for functional testing. It provides you with a comprehensive set of features for functional testing - such as response processing allowing to check correctness of server responses. You can either simply re-use the scripts created for load testing in functional testing - without modifications, or easily create new ones. Thus you can create comprehensive sets of common test cases for your project.

  • Convenient User Interface
    The integrated testing environment offered by Web Roller allows to perform all testing tasks (from initial design of test sequences to final reporting) without leaving the application. Project-based script editor is provided for easy script management. The ability to run test scripts and test projects from Win32 console adds to Web Roller flexibility.


Glossary

Here we list the most important terms required for proper understanding of Web Roller operation:

  • Function - an elementary building block of a request parameter. A single parameter can be built from multiple concatenated functions.
  • Parameter - a single <name>=<value> pair in a test script. There are global parameters and request parameters. Request parameters are built from functions.
  • Modifier - a value (or a range in the format [from..to:step]) used to modify a global parameter of the test script during the test run.
  • Request - a single HTTP request described in terms of Web Roller. Web Roller builds the request string from parameters and sends it to the Web server.
  • Test sequence - a linear sequence of requests. A part of a test script.
  • Test script - a set of global test parameters, requests and test sequences. Equivalent of a single test script file.
  • Test script run - a test script plus a set of global parameters altered by modifiers. Equivalent of a single line in the [runoptions] section of a test project file.
  • Test project - a set of information about testing automation: which test script files to run, in which order, and how the global parameters of a test script are modified during the script run. Equivalent of a single test project file.

  • Test script file - a file with a TST extension, equivalent of a single test script.
  • Test project file - a file with a WPJ extension, equivalent of a single test project.


Installation and configuration

To ensure that Web Rollermeets the demands of the users, we keep up the work and frequently update the application. Check for the latest installation packages and updates at the Web Roller Downloads page.

The Web Roller settings can be configured using the Edit menu. In the Settings dialog you can set up the following parameters:

Proxy settings:

  • Proxy host - the host name for Web Roller built-in proxy engine. Defaults to localhost.
  • Proxy port - built-in proxy port number. Defaults to 8001.
  • Try to disable IE caching checkbox - turn this on if you want to avoid usage of locally stored files by Internet Explorer. With this setting on, Internet Explorer will always try to get the requested resource from Internet, no matter whether there already is a copy in the local IE file cache.

Reporter:

  • Max requests in graph - allows to set a limit for the amount of requests used for drawing a single graph. A great amount of charted request may lead to a messy, hardly understandable chart. Default - 50000.
  • If exceed: select what to do if the requests limit is exceeded. Either of Do not draw graph or Warn.

Runner:

  • Beep when finished checkbox: enable this if you want to hear an audible signal after completing all scheduled test runs.

Record:

  • Filters - the list of recorder filters. You can set the recorder to omit the requests including certain text strings (e.g., eliminate Flash clips by adding a filter for the .SWF file extension).
  • - add filter string;
  • - delete selected filter string;
  • Enable recorder filters checkbox - toggle the use of filters listed above;
  • Notify about script creation checkbox - enables an additional alert when you export captured requests from the Record tab. Each time you press the Add to Script button, Web Roller will display the «Requests has been added to script» message. Use the Don't show this message again checkbox if you want to disable further notifications.

General:

  • Reopen last project checkbox - tells Web Roller to remember the last project file and automatically open it at next program start.


Testing process

For detailed step-by-step instructions on designing your own test process please refer to the online manual on the Web Roller homepage.


File menu

From the File menu you manage (open/close etc.) test projects (.WPJ files) and individual test scripts (.TST files). The following commands are available:

  • New project - creates an empty project;
  • New script - creates an empty test script;
  • Open script - opens an existing test script file;
  • Save script - saves the current test script file;
  • Save script as - saves the current test script under a different name;
  • Open project - opens an existing project file;
  • Save project - saves the current project file (and all related script files);
  • Save project as - saves the current project under a different name;
  • Quit - exits Web Roller.


Edit menu

The Edit menu allows to edit global Web Roller preferences and to remove unneeded test scripts from your project:

  • Remove selected script - removes the selected script from the current project.
  • Settings - the list of global Web Roller settings.

Proxy settings:

  • Proxy host - the host name for Web Roller built-in proxy engine. Defaults to localhost.
  • Proxy port - built-in proxy port number. Defaults to 8001.
  • Try to disable IE caching checkbox - turn this on if you want to avoid usage of locally stored files by Internet Explorer. With this setting on, it will always try to get the requested resource from Internet, no matter whether there already is a copy in the local IE file cache.

Reporter:

  • Max requests in graph - allows to set a limit for the amount of requests used for drawing a single graph. A great amount of charted request may lead to a messy, hardly understandable image. Default - 50000.
  • If exceed: select what to do if the requests limit is exceeded. Either of Do not draw graph or Warn.

Runner:

  • Beep when finished checkbox: enable if you want to hear an audible signal after completing all scheduled test runs.

Record:

  • Filters - the list of recorder filters. You can set the recorder to omit the requests including certain text strings (e.g., eliminate Flash animations by adding a filter for the .SWF file extension).
  • - add filter string;
  • - delete selected filter string;
  • Enable recorder filters checkbox - toggle the use of filters listed above;
  • Notify about script creation checkbox - enables an additional alert when you export captured requests from the Record tab. Each time you press the Add to Script button, Web Roller will display the «Requests has been added to script» message. Use the Don't show this message again checkbox if you want to disable further notifications.

General:

  • Reopen last project checkbox - tells Web Roller to remember the last project and automatically open it at next program start.

- save the changes made and return to Web Roller main screen;

- discard changes and return to Web Roller main screen.


Help menu

The Help menu provides a link to the Help system and displays information about the application version/build.

  • Help Topics - displays this Help;
  • About - displays program version/build and copyright info.


Toolbar

The buttons on the Toolbar (below the menu line) duplicate most operations of the File menu. The following file operations can be launched by Toolbar buttons:

- create a new (blank) test project;

- open an existing test project (.WPJ) file;

- save the current test project;

- create a new (blank) test script;

- open an existing test script (.TST) file;
- save the current test script;

- remove current (selected) script from the current project;


Edit Tab

Edit

The Edit tab provides access to the set of parameters for the current test script (equivalent of a single .TST file). The parameters are grouped in five subsections: Record, Settings, Requests, Sequences and Source.

The left pane of the Edit tab provides access to all scripts in the current project. You can select an active script, add a new script (right-click on empty space and select New) or remove a script (right-click a script and select Remove).


Edit - Record

The Record tab captures HTTP requests from MS Internet Explorer while browsing pages and converts them into requests for Web Roller. To achieve this, upon each start Web Roller launches a small transparent proxy engine which listens for HTTP requests on port number specified in Settings, and modifies the MS Internet Explorer proxy settings to make it use this built-in proxy. The Internet Explorer proxy settings are restored to the original values when you exit Web Roller. You can also restore the proxy settings manually (see below).

You can use Web Roller to capture requests from other browsers as well. In this case you will have to manually set the proxy server name and proxy port of your specific browser to the same values that are set in the Settings dialog. For more details on how to do this please refer to your browser's Help.

Each of the captured HTTP requests is displayed in the Log Area. Thus all user activity on a web site is decomposed into separate simple requests. The desired set of requests can be exported as a set of requests which will be listed in the Requests section.

Controls:

- starts recording browser requests;
- stops recording browser requests;
- sets the IE global proxy settings to the values from the Settings dialog;
- resets the IE global proxy settings to original values;
- exports the requests listed in the Log Area to the active test script.

All captured requests are displayed as separate lines in the Log Area (request method, server and URI are listed). Before adding requests to the active script, you can filter the contents of the Log Area using file extensions: right-click a request line and select one of the following:

  • Filter out selected type(s) - to delete the highlighted type of requests from the log;
  • Leave only selected type(s) - to leave only the highlighted type of requests in the log;
  • Delete selection - to delete selected line(s).


Edit - Settings

The Settings tab consists of the following components:

  • The parameters pane, listing the setting names and values. The settings are grouped in four subsections: Globals, Scripting, Runtime and Output. The settings absent in the script source and set to default values are displayed in gray. See Syntax of Global Parameters for a detailed description of each setting.

    • To switch the value of a Yes/No setting, double-click on its name (or highlight the setting name and hit on Space).
    • To enter/edit values for Integer or String value parameters, click the parameter name and enter/edit the value in the editbox appearing in the Value column.

  • The Includes: pane listing external .TST files included in the current test script.

    • To add a file, right-click on an empty area in the pane and select Add . Then select .TST files using the standard Open File dialog.
    • To remove a file, right-click on its name and select Remove.

  • The Headers pane listing additional fields for the HTTP request header (UserAgent etc.). This is a pure text editing field. List additional request headers one per line, e.g.:

    UserAgent=Mozilla/4.0 (compatible; MSIE 4.01; Windows 95)
    Referrer=http://www.microsoft.com/


Edit - Requests

The Requests tab displays the Requests pane with the list of requests, the Request parameters pane with the list of parameters for the currently selected request and the controls to manage the behavior of separate functions used within the parameter, along with individual request settings.

Requests management
Highlight the desired request in the Request name pane, then press:

- to add a new request (or right-click on the pane and select Add);
- to rename a request (or right-click the desired request and select Rename);
- to delete a request (or right-click the desired request and select Delete).

Request parameters management
Highlight the desired parameter in the Request parameters pane, then press:

- to add a new parameter (or right-click on the pane and select Add);
- to rename a parameter (or right-click the desired parameter and select Rename);
- to delete a parameter (or right-click the desired parameter and select Delete).

Concatenation of functions
Each request parameter is composed of simple building elements - functions (see also in Syntax of request parameters). You can concatenate the results of multiple functions to build the value of a single parameter. The order of functions used in the parameter is controlled by the uppermost input field on the right side - it lists the order and the types of the components building up the current parameter. You can add and delete components or change their order in the parameter value string - use the following buttons:

- to add a component;
- to delete a component;
- to move the highlighted component one position up;
- to move the highlighted component one position down.

Function types and arguments
The Parameter type field controls what function will be used for the current component (the one selected in the parameter value string). The following function types are available:

  • Static text - plain text values;
  • Numeric random - generates a random integer in the range from - to, with an optional prefix.
  • Alphanumeric random - generates a random sequence of alphanumeric characters with length ranging as set by the from - to limits.
  • Random from list - substitutes a random value from the list. List one value per line in the List textarea.
  • Sequence - generates an increasing sequence (step=1) starting from Start (default - 0), with an optional prefix.
  • URL encoded - searches the response to previous request for a GET request with Name as one of parameters. The value of this parameter will be returned; if Name is empty or not found, the name of the target parameter will be used instead.
  • Hidden - searches the response to previous request for a hidden construction with <.... name=”Name” value=”{value}”>. Returns {value}.
  • RegExp - performs a match of the result of previous request against RegExp (see Regular Expressions). Returns the construction set in Replace. $0-$9 can be used in Replace to substitute blocks of RegExp delimited by parentheses. $0 stands for the whole match.
  • ClId - generates a string composed of Prefix plus the ordinal number of the current Virtual Client plus Postfix. Useful if you need a value which is different for different clients, but stays unchanged for the same client.
  • List - substitutes values from the list in the order they are listed, in a cycle. List one value per line in the List textarea.

You can use the following buttons:

- to copy the type and settings of the current function;

- to paste previously copied settings into the current function (pasting will also change the function type);

- to apply the modifications made to functions/parameters (the changes will be written to the script source in the Source tab).

Request properties
The Request properties area controls the request-specific parameters. Check the respective checkboxes to use modified parameters for the current request; uncheck to leave the global parameters active (see Syntax of Global Parameters):

  • Method - HTTP request method (GET/POST);
  • Delay - delay inserted after each request before starting the next one (milliseconds);
  • URI - request URI;
  • Server - specify a different server name or IP for current request;
  • Port - specify a different server port;
  • Fail RE - If the regular expression is matched in the response, the request is considered failed (see also $fail= in Syntax of request parameters).

- applies the modifications made to Request properties (the changes will be committed to the script source in the Source tab);


Edit - Sequences

The Sequences tab displays two panes with requests. The left pane (Requests:) lists all requests defined within the current test. The right pane (Sequences:) lists the requests currently present in either of [InitSeq], [TestSeq] and [FinalSeq] sections (see Test Scripts for an explanation of sequence sections). Use the buttons listed below:

- to switch to [InitSeq];

- to switch to [TestSeq];

- to switch to [FinalSeq].

The button of the currently selected sequence is grayed.

You can add and remove requests to/from different sequences, as well as reorder and duplicate requests within a sequence. Select an appropriate sequence (as described above), highlight the desired request and press:

- to add a request to the current sequence;

- to remove a request from the current sequence;

- to move the highlighted request one position up in the sequence;

- to move the highlighted request one position down in the sequence;

- to duplicate the highlighted request (the new request will be appended at the end of sequence).


Edit - Source

The Source tab provides the advanced users with the ability to manually edit the settings of a single test. It is actually a view to the resulting code of the current test script. You edit the code just as if you were editing the .TST file in a regular text editor.

For more details on test script syntax please refer to Syntax Reference.


Run

The Run tab displays two panes.

The Tests ready for execution pane (left) lists script runs of the current project. A script run is composed from the data of the source test script file and the set of modifiers (which alter the script's global parameters for this run). To use the same test script several times with a different set of modifiers, you should create multiple entries (script runs) in this list using the same source test script.
The whole script run string is displayed in the pane, i.e.: full path to the source script file plus the complete set of modifiers (as it appears in a Test Project file).
The buttons above the list of script runs allow to:

- add a new script run to the test project (the respective script file should be already loaded in the current project). Opens a blank Run dialog

- edit script run parameters (or right-click the desired script run and select Edit). Opens the Run dialog with the current script run.

- delete a script run from the project (or right-click the desired script run and select Delete).

The right pane (Execution log) displays short messages about the status of the currently running scripts. If Web Roller detects excessive CPU usage (over 90%) or massive HDD transfers during the test run, it will alert you with respective messages about possible loss of test precision.

The progress of the currently running script is displayed in the Web Roller status line (bottom of the application) as current request number vs. total amount of requests to run (both numeric values and a progress bar).

The buttons above the log area allow to:

- run all entries listed in the Tests ready for execution pane;

- run only the selected entries (use Shift or Control keys to select multiple entries);

- stop all script runs (both currently running and scheduled to run);

- stop the currently running script and switch to the next one scheduled.


Run dialog

The Run dialog allows to modify the settings of a single script run in the project (see also the Glossary). The following controls are available:

  • Select script combobox: a list of full paths to script files included in the current project. Here you select which test script to run and modify.
  • Select parameter combobox: a list of all parameters from the [General] section of the script.
  • Modifiers. You can set any of the script's global parameters to a specific value (other than initially specified in the script), or make numeric parameters cycle from one value to another with a desired step:
    • Select set to from the combobox and enter a new value for a desired parameter, or:
    • Select change from and enter the start value, end value and the step to make the selected parameter cycle through these values with the desired step.
  • Redefined parameters area: displays the set of modifiers for the current script run (in the order they will be saved in the test project file). The contents of this area is, actually, an alternative view of a single script run (hwich is equivalent to a single line from the [runoptions] section of the test project file). Each line in the window stands for a single modifier. If the same parameter was redefined multiple times (i.e., it occurs in several lines), the latest (lowest) setting will override the previous ones.
    See Test Projects for details on modifiers syntax and structure of test project (.WPJ) files.

- add a new parameter (with the currently set modifiers) to the current script run;

- edit the highlighted parameter (or right-click the desired line and select Edit);

- delete a parameter from the script run (or right-click the desired line and select Delete).

- confirm changes and exit the Run dialog;

- discard changes and exit the Run dialog.


Report

The Report tab displays the results of your test runs. You can view the results as text (report source) using the Report tab, or as graphs visualizing different aspects of the test run - using the Graph tab. Each separate request from the script is pictured as a separate data line on a graph. The legend to the right of the graph window lists color codes and names for different requests.

You can select which test run to visualize - use the leftmost combobox with test run summaries. For test runs cycling through numeric parameters, you can select different executions. E.g., if you cycled the amount of clients for the script yourtest.tst from 10 to 30 with a step of 10, you will be able to select one of the following lines:

yourtest.tst|”clients=[10..30:10]” - clients=10
yourtest.tst|”clients=[10..30:10]” - clients=20
yourtest.tst|”clients=[10..30:10]” - clients=30

The rightmost combobox selects the graph type. The following graphs can be built:

  • Request per second - average amount of requests per second in the cycle. Horizontal axis - cycled parameter (for the example above - three points with 10, 20 and 30 virtual clients);
  • Cycle minimum - minimum response time in the cycle (horizontal axis - cycled parameter);
  • Cycle average - average response time (horizontal axis - cycled parameter);
  • Cycle maximum - minimum response time (horizontal axis - cycled parameter);
  • Minimum - minimum response time for the selected line. Horizontal axis - request sequential number (as set in the [General] section);
  • Average - average response time (horizontal axis - request sequential number);
  • Maximum - maximum response time (horizontal axis - request sequential number);
  • 1..N - response times for individual virtual clients (horizontal axis - request sequential number);

Note that the first 4 graph types are available only for runs with cycling numeric parameters.

- clears the results of the last testing, leaving the Report tab empty;

- opens the Save Report Dialog.


Save report

Report - Save Report dialog

The Save Report dialog allows to save the numeric results and graphs from the current report as a single HTML page. It is composed of the following controls:

  • Test executions pane: lists all test runs for which you can save a report . You can only select a single line.
  • Graphs pane: lists the available graph types for the selected test run. Highlight the graphs you want to embed into your report (use Shift or Control keys to select multiple entries).
  • Image size combobox: select the appropriate size for you report's graph images.
  • Background color: select the background for the resulting HTML document and graphs.
  • Save text report checkbox: check if you want to save textual data along with graphs.

The .JPG graphic files required to display the HTML report will be placed in the same folder with the report itself.

- opens the standard Save File dialog for the resulting report;

- cancel exporting report data.


Recording a Test

You can use the Record tab to capture actual HTTP requests from MS Internet Explorer while you browse pages and then convert these into series of requests for Web Roller. To achieve this, upon each start Web Roller launches a small transparent proxy engine which listens for HTTP requests on port number specified in Settings, and modifies the MS Internet Explorer proxy settings to make it use this built-in proxy. The Internet Explorer proxy settings are restored to the original values when you exit Web Roller. You can also restore the proxy settings manually (see below). You can as well use Web Roller to capture requests from other browsers, but in this case you will have to manually set the proxy server name and proxy port of your specific browser to the values that are set in the Settings dialog. For more details on how to do this please refer to your browser's Help.

You can perform the required actions in the browser window (navigate sites, go to different pages, fill in and submit forms and so on). Meanwhile all HTTP requests issued by the browser are logged by Web Roller's built-in proxy server. When the recording is over, the log is exported into the test script (as a set of requests). The resulting test script can then be modified using standard methods.

The sequence of actions to record the test script in this case is as follows:

  1. Launch Web Roller and switch to the Edit-Record tab;
  2. Press the Set IE proxy settingsbutton. A blank browser window (MSIE) should appear.
  3. Press the Start Recording button and record the desired sequence of actions: In the browser window, perform the sequence of actions to be automated. As the sequence is recorded, the log of requests sent to the Web server is displayed in Web Roller's window.

    4. Note: the log can contain requests for files which are parts of the target page design: graphics (*.jpg, *.gif, *.png and so on), flash clips (*.swf), stylesheets (*.css) and other elements. If they are not critical for your testing purposes (e.g. you intend to measure the performance of an ASP script on the server side at different client loads), you can exclude these files from your test script by turning on the filtering of specific content type(s) (see Settings dialog) or by selecting the unnecessary elements in the log window (right-click the element) and deleting the selected files or types of files from the log.

  4. To stop recording user actions press the Stop Recording button.
  5. Convert the log to a test script. If you press the Add To Script button, all HTTP requests displayed in the log window will be exported (in the appropriate format) into the current test script. After generating the test script from the log, Web Roller automatically switches to the Requests tab.


Secure content recording

If the Secure parameter is set to 1, it turns on the use of HTTPS. You should understand that when you are testing an application using HTTPS it is impossible to record a test script by capturing the sequence of user actions in the browser. This is due to the fact that requests in this case are encrypted and Web Roller is unable to decrypt them. The test scripts for such a case can be developed using one of two possible alternatives:

During development of test scripts switch the tested application (if possible) to HTTP (instead of HTTPS). Record the sequence of user browser actions using the standard procedure, debug the testing scripts if required. After developing the test scripts, switch the application back to HTTPS and set the Secure option in the test scripts to 1. Develop the scripts manually or within the IDE (refer to Manual development of test scripts).


Working wht redirects

Working with redirects

Usually when test scripts with redirects are recorded, the redirects are not included in Web Roller's proxy log. But if “meta http-equiv” redirects are used, then the situation is possible when the log will include both the initial pages (containing redirects) and the redirects themselves. In this cases the extra redirects should be removed from the log manually.


Working with session variables

A typical mistake when automating scripts using session variables is using the static value for the session variable (obtained by recording the sequence of HTTP requests with the Record function) while these session variables are unique for each session; thus an attempt of running such a test script will lead to errors (since other sessions will generate different values of session variables). To provide proper functioning of test scripts in cases like this you need to extract respective session variables from the responses to previous requests (see the description of $UrlParam, $Hidden, $RegExp).


Manual Test Creation

Manual development of test scripts is suitable for advanced users of the program. The method is applicable only when the exact structure (and composition) of the HTTP request is known. The development itself can be carried out using either of the two methods described below (or the combination of both methods).

Writing and editing the source code

Switch to the Edit-Source tab: here you may author the script source according to general syntax rules (see in Syntax reference).

Developing the script in the IDE (Integrated Development Editor)

Switch to the Edit-Requests tab and proceed as specified in its description (see also Edit-Settings, Edit-Sequence for a more detailed description of IDE controls for this tab and section).

If we do not know the structure of the HTTP request a priori, we cannot use the methods described in this section, since we do not know neither the names nor the values of request parameters (pairs parameter1=value1&parameter2= value2).


Global settings

The [General] section defines global settings of the test, default values of certain parameters and options controlling the behavior of the virtual clients and the format of reports. These global settings will be used if not overridden by the script run parameters or local HTTP request declarations.

This section provides detailed descriptions of the parameters grouped by subsections related to different aspects of testing.


Test volume

Clients

Syntax example:

clients=5

Sets the amount of virtual clients.

Requests

Syntax example:

requests=10

Sets the number of runs of the main cycle [TestSeq] for each virtual client. E.g. if the settings are Clients=10, Requests=20 and there is a single HTTP request declared in [TestSeq], the overall number of [TestSeq] cycle runs performed by all virtual clients will make 10x20=200, thus 200 HTTP requests will be sent to the server. If there are 5 HTTP requests declared in [TestSeq], then the overall number of [TestSeq] runs in this test will remain the same - 10x20=200, but the amount of HTTP requests sent to the server will make 10x20x5=1000.

MaxRequests

Limits the maximum overall amount of requests made by all virtual clients. When this number is reached, all virtual clients will be stopped - disregarding the amount of requests made by each particular client.

This setting can be useful when measuring the precise value of Requests per second. Usually the chart of server load chart with multiple virtual clients looks as follows:

The reason of the stepwise character of the load rise at the initial stage was described earlier (see Delaystart). The reason of the stepwise character of the final stage is similar: it is both the fact that not all virtual clients start at the same time, and the various work times of particular clients (which can differ significantly).

Web Roller calculates the Requests per second value in the following manner:

Requests per second = Total requests made / Working period

where:

Working period - the overall time spent for processing all requests - from the time of sending the very first request till the end of processing the last one;
Total requests made – the overall number of requests made. Calculated as:

Clients x Requests x (Total number of requests in [TestSeq] section)

Thus the Requests per second value is measured incorrectly, since constant load is not provided during the Working period. You cannot provide accurate conclusions about application performance tested under a specific load (200 virtual clients in our case).

The method of avoiding the starting "steps" is described in Delaystart. The MaxRequests setting allows in turn to avoid the final "steps". The required actions are:

  • Set an excessive value of Requests (e.g. 1 000 000);
  • Set the MaxRequests value so that the amount is sufficient for correct calculation of Requests per second (i.e. if we set MaxRequests=10, then the value of Requests per second calculated using only 10 requests can be claimed to have a great calculation error and thus cannot be used as a realistic estimate; if we set MaxRequests=10000, then you can consider the calculation error to be acceptable).

It also a good idea to consider also the running time of the script when setting the MaxRequests value - e.g. if the script successfully performs 10000 requests in 5 seconds, than the Requests per second will be calculated with a noticeable error).

Attention!

You need to remember that Requests sets the amount of runs for the [TestSeq] sequence which can include multiple HTTP requests, while MaxRequests sets the maximum amount of processed HTTP requests for ALL clients. The MaxRequests value should be significantly less than the value of Total requests made (see above for the calculation of this value), calculated for the case when MaxRequests equals to 0 (i.e. not used).

Maxtime

The maximum time for running the test (seconds). The test will run until all requests will be processed, or until the run time exceeds Maxtime.

Caution!!!

For this setting to be active, the amount of requests should be big enough so that all requests are not processed before Maxtime elapses.

Abortseq

Determines whether to abort the current sequence [TestSeq] for all virtual clients if Maxtime is reached, or to wait for normal sequence termination.


Server

Server

Syntax examples:

server=www.webapplicationstesting.com
 server=194.149.225.75

Name or IP address of the server being tested. E.g., www.webapplicationstesting.com or 194.149.225.75

Port

Syntax example:

port=5555

The server port to which the requests are sent.

Httpver

The version of HTTP protocol used by the virtual clients.

1 – Version 1.1;
0 – Version 1.0.

Method

Syntax example:

method=GET

The default method of HTTP request (GET or POST) used in the declared requests (see also $method).

Uri

Syntax example:

uri=/categories/category1.html

Default URI (see also $URI).

Keepalive

Determines whether to use keep-alive requests (see also $keepalive):

1 – make all requests keep-alive;
0 – make all requests non-keep-alive.


Proxy

Proxy

All HTTP requests generated by Web Roller will be made through the specified proxy server. Provide host:port.

Proxyemulation

Toggles the use of additional fields of the HTTP header for imitating different users working through a proxy server. Useful to emulate requests coming from multiple computers.

0 – do not use emulation;
1 – use emulation.

Proxymask

IP address mask used in Proxyemulation:

$C1 – low byte of the client number;
$C2 – high byte of the client number;
$R1 – low byte of the request number;
$R2 - high byte of the request number.

All values span form 1 to 254. To emulate requests coming from unique IP's, set the mask to:

$C2.$C1.$R2.$R1

Obviously if the amount of requests exceeds 64262, the $Rx values will be repeated. You can use a static value instead of $C2 if the amount of clients is lower than 253.


Redirects

Noredir

Handling of redirects:

0 – redirects are processed;
1 - redirects are ignored.

By default Web Roller handles redirects. Thus, if a redirect is obtained in response to a HTTP request, Web Roller will make an additional request to the location specified by the redirect. If yet another redirect is obtained, another request will be made, and so on. By default the total time of processing all redirects in a series is accounted as the response time for the current request. If some of the requests in this chain of redirects results in an error, the request is accounted as failed and the respective error code is written to the test report.

Sumredirecttime

Defines whether to add up request times for redirected requests:

0 – redirect times are not added, only the response time of the very first request is written to the report;
1 - redirect times are added, the total for all redirects is written in the report.


Secure content

Secure

If equal to 1, turns on the use of HTTPS. You should understand that when you are testing an application using HTTPS it is impossible to record a test script by capturing the sequence of user actions in the browser. This is due to the fact that requests in this case are encrypted and Web Roller is unable to decrypt them. The test scripts for such a case can be developed using one of two possible alternatives:

During development of test scripts switch the tested application (if possible) to HTTP (instead of HTTPS). Record the sequence of user browser actions using the standard procedure, debug the testing scripts if required. After developing the test scripts, switch the application back to HTTPS and set the Secure option in the test scripts to 1. Develop the scripts manually or within the IDE (refer to Manual - “Manual development of test scripts”).

Authorization

Syntax:

authorization=login:pass

Basic authorization. Substitutes the specified login:pass in requests requiring basic authorization.


Saving system resources

NoDetails

Toggles whether to write to the report the time of individual requests or only the summary statistics. It is a good idea to turn off the details when measuring overall performance, i.e. when a precise value of Requests per second is required.

NoWait

Turns off the timeout check. Allows to provide significant server load using a single workstation, thanks to saved resources. Consequently, can lead to hang-ups of virtual clients in case of big amount of stuck requests (which would be normally closed using the timeout check).


Test start

Delaystart

Delay between starting times of different virtual clients. By default, if the number of virtual clients is big enough, the initial load on the server rises, but not instantly. E.g. if Clients=200, the chart of virtual clients' start times will look as follows,

potentially leading to poorer precision of the measured Requests per second value. If you set Delaystart to 0, all virtual clients will start exactly at the same time:


Scripting

Script

Defines the use of test script (inline or external). Inline script should be enclosed in the [script]...[/script] pair of tags.


Connection speed

MaxSpeed

Sets the network connection bandwidth limit for virtual users (in kbps). Useful to emulate e.g. a modem connection.


Errors

IgnoreErrors

Defines whether to abort the sequence if an error was detected for a request. Useful for “negative” tests, i.e. when an error is an expected result.

NoWait

Turns off the timeout check. Allows to provide significant server load using a single workstation, thanks to saved resources. Consequently, can lead to hang-ups of virtual clients in case of a big amount of stuck requests (which would be normally closed using the timeout check).


Timing mode

TimingMode

The mode of time measurement:

0 – WebTransaction, the time period between the last request byte sent and the last response byte received;
1 – ResponseTime, the server response time. Works only under HTTP/1.1. Time period between the last request byte sent and the first response byte received;
2 – RequestTime, full request processing time. Time period between the first request byte sent and the last response byte received.


Full log recording

Log

All the run results are written to the log file. Each virtual client produces a separate log file.

The comprehensive log of the Web Roller client run is created, including all requests, responses, redirects and the returned HTML code. If the response is obtained in chunked transfer encoding, it will be converted to normal presentation.

Execution Log

If in the [General] section Log=1, all the results of operation are logged to a file. Each virtual client generates a separate log file. The complete log of the Web Roller client operation is written to this file. This includes all requests, responses, redirects and the HTML code returned.

A test script like this:

[General]
clients=1
requests=2
server=kort
log=1

[TestSeq]
Req1
Req2

[Req1]
$uri=/
$method=GET

[Req2]
$uri=/view.pl
$method=GET

will result in the following log file:

----- [TestSeq]: Req1(0) ----- | Separator line: pass #0 of [TestSeq], request Req1
Request: | Header of the HTTP request sent
GET / HTTP/1.1
Host: kort
Connection: close
Response: | Header of the server response.
HTTP/1.1 200 OK
Date: Sun, 21 Apr 2002 16:30:45 GMT
Server: Apache/1.3.12 (Win32)
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> | HTML code of response
<html>
</body>
..
..
</body>
</html>
----- [TestSeq]: Req2 (0) ----- | Separator line: pass #0 of [TestSeq], request Req2
Request: |
GET /view.pl HTTP/1.1 |
Host: kort |
Referrer: http://kort/ |
Connection: close |
Response: |
HTTP/1.1 200 OK |
Date: Sun, 21 Apr 2002 16:30:45 GMT |
Server: Apache/1.3.12 (Win32) |
Connection: close |
Transfer-Encoding: chunked |
Content-Type: text/plain |
<html> |
</body> |
.. |
.. |
</body> |
</html> |
----- [TestSeq]: Req1(1) ----- | Separator line: pass #1 of [TestSeq], request Req1
Request: |
GET / HTTP/1.1 |
Host: kort |
Referrer: http://kort/view.pl |
Connection: close |
Response: |
HTTP/1.1 200 OK |
Date: Sun, 21 Apr 2002 16:30:45 GMT |
Server: Apache/1.3.12 (Win32) |
Connection: close |
Transfer-Encoding: chunked |
Content-Type: text/html |
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> |
<html> |
</body> |
.. |
.. |
</body> |
</html> |
----- [TestSeq]: Req2(1) ----- | Separator line: pass #1 of [TestSeq], request Req2
Request: |
GET /view.pl HTTP/1.1 |
Host: kort |
Referrer: http://kort/ |
Connection: close |
Response: |
HTTP/1.1 200 OK |
Date: Sun, 21 Apr 2002 16:30:45 GMT |
Server: Apache/1.3.12 (Win32) |
Connection: close |
Transfer-Encoding: chunked |
Content-Type: text/plain |
<html> |
</body> |
.. |
.. |
</body> |
</html> |


Report configuration - general parameters

Report configuration: general parameters

Outfile

The name of the test run textual report file. Since MS Excel™ accepts data in Tab-Separated format used for the resulting report, the generated file has the xls extension, to avoid unneeded renaming.

Outfile can be set both in plain format and using the naming templates. The following macros can be used:

%s - the server
      setting of the General section;
%p - the port setting
      of the General section;
%r - the requests
      setting of the General section;
%c – the clients
      setting of the General section;
%f - the full name of the test script file;
%n - the name of the test script file (no extension);

Current date/time macros:

%y - year;
%m – month;
%d – day;
%t – hhmmss;

%% - the % character.

The default template for the file name is %n-%s-%cx%r.xls

Log

All the run results are written to the log file. Each virtual client produces a separate log file.

The comprehensive log of the Web Roller client run is created, including all requests, responses, redirects and the returned HTML code. If the response is obtained in chunked transfer encoding, it will be converted to normal presentation.

TimingMode

The mode of time measurement:

0 – WebTransaction, the time period between the last request byte sent and the last response byte received;
1 – ResponseTime, the server response time. Works only under HTTP/1.1. Time period between the last request byte sent and the first response byte received;
2 – RequestTime, full request processing time. Time period between the first request byte sent and the last response byte received.

NoDetails

Toggles whether to write to the report the time of individual requests or only the summary statistics. It is a good idea to turn off the details when measuring overall performance, i.e. when a precise value of Requests per second is required.

Appendresult

Upon launching the test script the results are either appended to the existing report file, or the existing file is replaced with the new one each time the test script is launched.

Showbadresults

Toggles whether to log the requests which lead to errors.

1 – write to the log;
0 – do not write to the log.

Sumredirecttime

Defines whether to add up request times for redirected requests:

0 – redirect times are not added, only the response time of the very first request is written to the report;
1 - redirect times are added, the total for all redirects is written in the report.


Report configuration - report header parameters

Report configuration: report header parameters

Printdate, Reportinfo

Toggle whether to print supplementary test info in the beginning of the report file - including the run date, amount of clients and requests, the user and workstation names of the computer which ran the test. If Projectname and Description are specified, these will be listed as well.

Projectname

Information string. Used mainly for Reportinfo.

Description

Information string. Used mainly for Reportinfo.


Report configuration - report body parameters

Report configuration: report body parameters

Round

Rounding precision.

10 – round to 0.1;
0 – no rounding, minimal possible value is 0.01 ms.

Timestamp

Toggles whether to write the starting and ending times for each request cycle to the report:

0 – do not write;
1 – write.

Locale

International settings for the report. E.g. to use “,” as the decimal separator you should set the locale to Russian. The default decimal separator is a period, setting locale to Russian switches to a comma.

Summarytimes

Defines whether to print Summary times in the report.

0 – do not print;
1 – print

You should not use Delay or other resource-consuming options (e.g. regular expressions) together with this option, otherwise the results will not be correct. Only the time period when all clients were launched and none of them has already terminated its run should be used for calculation of Summary times. If the amount of requests is not big enough, probably this requirement will not be met and the respective warning about results approximation will be added to the report.

ExactTimingInterval

Sets the value of minimum time period (milliseconds) for simultaneous work of all clients required to correctly estimate SummaryTimes.


Report configuration - error messages

Report configuration: error messages

Httperrorsymbol

A string (up to 8 characters) used in the report instead of response time if a HTTP error was detected (403, 404, 500 etc.). For instance, "HTTPERR".

Socketerrorsymbol

A string (up to 8 characters) used in the report instead of response time if a network (socket) error was detected.

Failerrorsymbol

A string (up to 8 characters) used in the report instead of response time if the $fail error condition was detected.

Timeouterrorsymbol

A string (up to 8 characters) used in the report instead of response time if the response time exceeded the Timeout value.


Request settings - local redefinition

Request settings: local redefinition

Each HTTP request declared in the test uses the sending parameters set in the [General] section. But these parameters can be modified when required, and made unique for each request. The essence of these local settings is the same as for the respective [General] section ones. The parameter names are obtained by adding a $ character to the name of the respective parameter from the [General] section ($URI, $Server, $Port and so on). The locally defined parameters have a higher priority then the global ones, i.e. the global values will be used only if no local parameters were set. Examine the following test script:

Example 1

[General]

Method=GET
Server=myserver
Port=5000
URI=/servlet
[Request1]
param1=value1
param2=value2

In this case Request1 will use the values of Server, Port, URI set globally (in the [General] section), since these are not locally redefined. The resulting HTTP request will look as follows:

GET /servlet?param1=value1&param2=value2 HTTP/1.1
Host: myserver:5000

Now let us redefine locally some parameters, and we will obtain

Example 2

[General]

Method=GET
Server=myserver
Port=5000
URI=/servlet

[Request1]

$Server=server2
$Port=8000
$URI=/servlet/view
param1=value1
param2=value2

Here the Request1 will use the locally redefined values - $URI, $Port. The resulting HTTP request will be:

GET /servlet/view?param1=value1&param2=value2 HTTP/1.1
Host: myserver:8000

Not all of the settings from the [General] section can be redefined locally. The parameters that can be redefined are listed below.

$Server

Syntax:

$Server={servername}

Locally defined name (or IP) of the server to send the request to.

$Port

Syntax:

$Port={portnumber}

Locally defined server port to send the request to.

$URI

Syntax:

$URI={uri|$href(text)|$action|$var(name,default)}

Locally defined request URI. Unlike the globally defined one, it can use either a static value or one of the following functions:

$href(text) – searches for the link with the specified text in the HTML code of response to the previous request and takes the URI from this link;

Example:

The HTML code of the response contains:

<A HREF="/manual_3.html">Preparing test scripts</a>

If we use in the subsequent request the $URI=$href(Preparing test scripts) function, we will finally obtain the actual URI of this link, which is equivalent to $URI=/manual_3.html.

$action – the first form in the respnonse to previous request is found and the URI is extracted from the form action.

Example:

The HTML code of the response contains:

<FORM NAME="search" METHOD="GET" ACTION="/cgi-bin/search.pl">
<INPUT TYPE="text" NAME="text" VALUE="" MAXLENGTH=80>
<INPUT TYPE="submit" VALUE="Start search">

If we use in the subsequent request the $URI=$action function, we will finally obtain the actual URI of this form action, which is equivalent to $URI=/cgi-bin/search.pl.

$var(name,default) – the URI value will be taken from the variable previously set by the $exportvar function. If the specified variable is undefined, the default value will be used.

$Method

Syntax:

$Method={GET|POST}

Request method.

$Keepalive

Syntax:

$Keepalive={0|1}

Toggles the use of keep-alive requests.

$Delay

Syntax:

$Delay={number}

Delay (in ms) after request execution.

$Fail

Syntax:

$Fail=[!]/regexp/

If the specified regular expression is matched in the HTML code of the response to the current request, the request will be accounted as failed. (see Failerrorsymbol). If you place the ! character (exclamation point) before the regular expression, the result of matching will be inverted, i.e. the request will be accounted as failed if the response HTML code DOES NOT contain the matching pattern.

Example:

The test script is:

[General]

requests=5
server=www.yahoo.com

[Request1]

$uri=/
$method=GET
$fail=/Yahoo/

The HTML code of the response contains:

<HTML><HEAD><TITLE>Yahoo!</TITLE>

If the request description contains $Fail=/Yahoo/, the request will be accounted as failed and the string specified by the Failerrorsymbol setting will be written to the report instead of request processing time. This setting is useful for functional testing.

$Exportvar

Syntax:

$Exportvar=(varname),/regexp/,{replacestring}

Defines a variable and its value (see also $Var, $RegExp). Extracts the pattern matching the regular expression from the current response HTML code and stores it in a variable named varname. $0-$9 metacharacters can be used in {replacestring} to specify the groups in the text matched by the regular expression.


Dynamic calculation of request parameters

In this section the term "parameters" is used to describe the parameters specified to the right of the “?” sign in the HTTP request string. The functions are used to define the values of HTTP request parameters and provide the ability to dynamically set the required values for the Param=Value pairs. If a function is used, the declaration of a parameter looks like this: Param=$function. The functions substitute the resulting values, with one exception being the $rxname function (see below).

Now let us examine the available functions and the resulting values that can be generated.

$NRand

Syntax:

ParamName=$NRand({from},{to}[,{prefix}])

A random integer from the [from,to] range; precede by a {prefix} (if one specified).

Example:

The test script is:

[General]
clients=1
requests=3
server=myserver
uri=/view

[Request1]

param=$NRand(1,20,ID)

If you run this script , three HTTP requests will be sent to the server:

http://myserver/view?param=ID5
http://myserver/view?param=ID12
http://myserver/view?param=ID2

The part of the param value after ID is generated randomly. If there is nothing specified instead of ID in the script, i.e. the script is as follows:

[General]

clients=1
requests=3
server=myserver
uri=/view

[Request1]
param=$NRand(1,20)

then three different requests will be sent to the server upon the script run:

http://myserver/view?param=5
http://myserver/view?param=12
http://myserver/view?param=2

$ARand

Syntax:

ParamName=$ARand({from},{to})

Random alphanumeric sequence with length in the range [from, to].

Example:

Launching the test script below:

[General]
clients=1
requests=3
server=myserver
uri=/view
.. ..

[Request1]

param=$ARand(1,20)

results in issuing three HTTP requests:

http://myserver/view?param=g
http://myserver/view?param=fndy5lgs5pck
http://myserver/view?param=xfglfgjl4asd098ljk6c

Here all values of param are randomly generated.

$LRand

Syntax:

ParamName=$LRand(value1[,{value2},…])

Random value from the list of values {value}.

Example:

Launching this test script:

[General]

clients=1
requests=3
server=myserver
uri=/view
.. ..

[Request1]

param=$LRand(test4,bobby,alpha)

Results in sending three HTTP requests to the server:

http://myserver/view?param=alpha
http://myserver/view?param=test4
http://myserver/view?param=bobby

$List

Syntax:

ParamName=$List(value1[,{value2},…])

Similar to $LRand, but the elements are used in the order they are listed (not randomly). The 1st request uses the 1st value from the list, the 2nd one uses the second value and so on. After reaching the last value the list is looped, starting again from the first listed value.

$Seq

Syntax:

ParamName=$Seq({prefix}[,{startvalue}[,{clientshift}]])

Augmenting sequence. Increases by 1 with each run of [TestSeq]. The initial value is {startvalue} (if specified, otherwize 0). If the {clientshift} is specified, then the starting value for different virtual clients will be shifted using this value. You can also specify several leading zeroes for the startvalue, so that the numeric component always keeps the same length.

Example 1:

The following test script:

[General]
clients=1
requests=3
server=myserver
uri=/view
.. ..

[Request1]

param=$Seq(test,2)

Results in sending three HTTP requests to the server:

http://myserver/view?param=test2
http://myserver/view?param=test3
http://myserver/view?param=test4

Example 2:

If the following test script is launched:

[General]

clients=3
requests=3
server=myserver
uri=/view
.. ..

[Request1]

param=$Seq(test,2,5)

the virtual client #1 will send three HTTP requests to the server:

http://myserver/view?param=test2
http://myserver/view?param=test3
http://myserver/view?param=test4

virtual client #2 will send three different HTTP requests:

http://myserver/view?param=test7
http://myserver/view?param=test8
http://myserver/view?param=test9

and virtual client #3, again, three different HTTP requests:

http://myserver/view?param=test12
http://myserver/view?param=test13
http://myserver/view?param=test14

$UrlParam

Syntax:

ParamName=$UrlParam[({name})]

The response to the previous request will be searched for a GET request containing a parameter with name {name}. The value of this parameter will be substituted. If {name} is not explicitly set, then the name of the target parameter (ParamName) will be used - i.e., Myparam=$UrlParam is completely equivalent to Myparam=$UrlParam(Myparam).

Example 1:

The HTML code of the response contains:

<A HREF=”http://myserver/view?SessionID=FCHHAB02DF&param=test111”</a>

Use of Myparam=$UrlParam(SessionID) in the next request will result in the actual value of Myparam=FCHHAB02DF.
This function is very useful when you need to use the parameter values generated by the server on-the-fly. This relates, e.g., to various Session variables.

Example 2:

You need to automate a simple scenario of navigating through a certain site using the respective HTTP requests:

1.User registration:

http://myserver/register?User=Alex&Pass=qwerty (a POST request)

2.After registration the user is redirected to the main category listing the links to all resources of the site. At this stage the SessionID session variable is generated. The user selected the Main category by clicking the respective link:

http://myserver/view?SessionID=FCHHAB02DF&Category=main

3.The user selects a link to a particular resource:

http://myserver/view?SessionID=FCHHAB02DF&Category=main&Item=67

If we record this sequence of HTTP request, it would result in the following test script:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/register
$Method=POST
User=Alex
Pass=qwerty

[Req2]
$Uri=/view
$Method=GET
SessionID=FCHHAB02DF
Category=main

[Req3]
$Uri=/view
$Method=GET
SessionID=FCHHAB02DF
Category=main
Item=67

But we will not be able to use this script - after sending Req2 an error will occur. The reason is that we try to use the old value of SessionID. Consequently, we need that each time after user registration the test script operates with a correct (currently generated) value of SessionID. We can modify the script to achieve this:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/register
$Method=POST
User=Alex
Pass=qwerty

[Req2]
$Uri=/view
$Method=GET
SessionID=$UrlParam(SessionID)
Category=main

[Req3]
$Uri=/view
$Method=GET
SessionID=$UrlParam(SessionID)
Category=main
Item=67

and thus solve the problems related to SessionID. Now before sending the Req2 request the $UrlParam function will parse the response (the returned HTML code) to request Req1, find the first link containing the required SessionID and substitute the actual value of SessionID into Req2 (in the Example this value is FCHHAB02DF).

$Hidden

Syntax:

ParamName=$Hidden[({name})]

The function searches for an element containing <.... name=”{name}” value=”{value}”> using the specified {name}. The found {value} will be used. If {name} is not specified explicitly, the name of the target parameter (ParamName) will be used. The syntax of this type is used typically to define input elements in HTML forms. The advantage of this function is that it allows to work e.g. with hidden form elements with dynamically generated values (again the Session variables).

Example:

The response to the previous HTTP request contains a form with a set of hidden elements:

<form action="/cgi-bin/select.pl" method="get">
<input type="text" name="index" size="25" maxlength="25">
<input type="submit" value="Search">
<input type="hidden" name="restrict1" value="yes">
<input type="hidden" name="restrict3" value="no">
....
....
</form>

and the values of these elements should be used in the subsequent HTTP request. Let us compose a test script for solving this problem:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/

[Req2]
$Uri=/cgi-bin/select.pl
$Method=GET
index=AnyText
restrict1=$Hidden
restrict3=$Hidden

In the test script above, before sending the Req2 request the $Hidden function will parse the HTML code of response to request Req1. For the restrict1 parameter the function will find the fragment <….name="restrict1" value=……>, for restrict3 - respectively, <….name="restrict3" value=……>. Respective values will be returned and used as values of parameters restrict1 and restrict3 in Req2.

$RegExp

Syntax:

ParamName=$RegExp(/{regexp}/,{replacestring})

The response to the previous request will be scanned for a match to the specified regular expression {regexp}, and the resulting function value will be built according to the search results and the {replacestring} syntax. The syntax is similar to Perl syntax (see description in Syntax Reference ->Regular Expressions).

Example:

You need to automate the submitting of a form with comboboxes. Only the description of combobox values are static (in this case - minimal, middle and maximum), but the values are generated by the server and not initially known. The HTML code of such a form may look like this:

<form action=”send” method=”post”>
.....
.....
<select name="weight">
<option value="20">minimal</option>
<option value="40">middle</option>
<option value="60">maximum</option>
</select>
</form>

Let us compose a test script for solving this problem:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/

[Req2]
$Uri=/send
$Method=POST
weight=$RegExp(/value=\"(\d)\">minimal</,$1)

Before sending the Req2 request $RegExp parses the HTML code of response to the Req1 request. For the weight parameter it will find the fragment <option value="20">minimal</option>, which matches the regular expression. The value 20 will be extracted and returned by the $RegExp function.

$RxName

Syntax:

$RxName(/{regexp}/,{replacestring},{value})

Modifies the parameter name according to the regular expression. No matter which parameter name was initially used in the script, it will be replaced by the result built according to {replacestring}. The rules are the same as for $regexp(). The {value} can be any correct value used in the request parameters, e.g. the ones listed above.

Example:

You need to submit a form for which the names of input elements are dynamically generated by the server, and thus initially unknown. The form is like follows:

<form action="send">
<input type="checkbox" name="Option124">Food
<input type="checkbox" name="Option78">Juice
<input type="checkbox" name="Option471">Biscuit
</form>

The test script for performing this task looks as follows:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/

[Req2]
$Uri=/send
AnyText=$rxname(/name="(Option\d+)">Juice/i,$1),on

Before sending the Req2 request the $RxName function parses the HTML code returned in response to Req1 and finds the code fragment name="Option78">Juice matching the regular expression. The value Option78 is extracted and used as the parameter name, the value of this parameter is then set to on.

$Var

Syntax:

ParamName=$Var({name}[{,default}])

Uses a variable set earlier by the $exportvar function. If the variable with name {name} is not defined, the {default} value will be used instead (if present).

Example:

Let us examine once again the example for the $RegExp function and solve the problem in a different manner. You need to automate the submitting of a form with comboboxes. Only the description of combobox values are static (in this case - minimal, middle and maximum), but the values are generated by the server and not initially known. The HTML of such a form may look like this:

<form action=”send” method=”post”>
.....
.....
<select name="weight">
<option value="20">minimal</option>
<option value="40">middle</option>
<option value="60">maximum</option>
</select>
</form>

Let us compose a test script for solving this problem:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/
$ExportVar=VarName,/value=\"(\d)\">minimal</,$1

[Req2]
$Uri=/send
$Method=POST
weight=$Var(VarName)

After sending the Req1 request the $ExportVar function parses the HTML code returned in response to Req1 and finds the code fragment <option value="20">minimal</option> which matches the regular expression. The value 20 is extracted and assigned to the VarName variable. Before sending the Req2 request the $Var function returns the value of the VarName variable (defined earlier), and assigns this value to the weight parameter.

All functions parsing the HTML code of the responses have one common limitation - they can operate only with the code of the previous request. Defining and using external variables with $ExportVar and $Var functions allows to overcome this limitation and transfer data between non-subsequent HTTP requests.

$File

Syntax:

$File({path filename}[{,mimetype}])

File upload. Multipart/form-data will be used. If {mimetype} is not explicitly specified, application/octet-stream will be used. Used to upload a file to the server, e.g. when sending e-mail messages with attachments via the Web interface of the mailing system.

Example:

Usually the code for file upload looks as follows:

<form method="post" action="upload" enctype="multipart/form-data">
<input type="file" name="filename">
</form>

If you need to upload the JPEG image image.jpg located in the C:\Graphics\Jpg\ folder, the test script will look as the one below:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/upload
$Method=POST
filename=$File(C:\Graphics\Jpg\image.jpg,image/pjpeg)

$Clid

Syntax:

ParamName=$Clid({prefix})

{prefix} plus the ordinal number of the virtual client. Can be used to create values which should be different for different clients, but the same for a single client.

$GVar

Syntax:

Paramname=$gvar(OptionName, DefaultValue)

Defines the parameter value when running as a Win32 console application (see Console mode).

OptionName – the name of command line option,
DefaultValue – default value of $gvar function.

Upon the launch of a test script woth a $gvar function the latter substitutes the value of the respective option from the command line (or the default value, if the requested OptionName was not found in the command line). Can be used for defining any parameters (or their components). CANNOT be used to define a parameter name (i.e., cannot stay to the left of the “=” sign).

Examples (correct use):

Server=$gvar(server, myserver)
$Server=$gvar(server, myserver)
param=$gvar(param, value1)
param=abc$gvar(param, value1)

Example (incorrect use):

$gvar(paramname, value)=value9


Regular expressions

  • Metacharacters
  • Quantifiers
  • Special Characters
  • Backreferences
  • Options

The Web Roller regular expressions syntax is almost identical to the regular Perl syntax. General syntax is:

/regexp/options

where regexp stands for the sequence of characters and metacharacters. Letters and numbers act as characters. All the rest potentially is metacharacters, since everything that is not a letter or a digit should be explicitly specified as a character.
Thus, to enter a non-alphanumeric character you will need to enter also a preceding \ (backslash).

Metacharacters

\ | - specifies that the next character will be treated as a character (not as a metacharacter).
. | - any character except for line feed. If there is an S in the options, then matches any character.
^ | - beginning of line. If the m option is specified, then the beginning of any line within the text is matched, otherwise - only the beginning of the whole text.
$ | - end of line. If the m option is specified, then the end of any line within the text is matched, otherwise - only the end of the whole text.
| | - alternative (OR). Matches either the expression before or the expression after |.
() | - grouping.
[] | - class of characters.

Grouping serves either for using backreferences or to specify the matching fragment within the parentheses for replacement.

A Class of characters determines lists or ranges of characters. It matches any of the characters listed within the [] (or within one of the listed ranges). Continuous ranges are specified as [a-z]. Generally you cannot use other metacharacters inside classes. Placing a ^ in the first position of a set inverts it (the set becomes negative). If you want to use the ^ as a character, it should either be not in the first position or be preceded by a backslash.

Quantifiers

Each character, group or class can be followed by one of quantifiers:

? | - greedy match (0 or 1 times).
?? | - non-greedy match (0 or 1 times).
* | - greedy match (0 or more times).
*? | - non-greedy match (0 or more times)
+ | - greedy match (1 or more times).
+? | - non-greedy match (1 or more times).
{n} | - match exactly n times.
{n,} | - greedy match (n or more times).
{n,}? | - non-greedy match (n or more times).
{n,m} | - greedy match (greater or equal than n, less or equal than m times).
{n,m}? | - non-greedy match (greater or equal than n, less or equal than m times).
Greedy and non-greedy quantifiers.

A greedy quantifier attempts to match as much of text as possible. Only if the next iteration is unsuccessful it "releases" the captured characters (this is called "rollback" - a rather time- and resource-consuming procedure). E.g. if we attempt to match the /A.*Z/ expression against the string:

AZXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX

then the .* will first capture all of the string and then will sequentially roll back until it finds the Z character.
If we use the non-greedy quantifier /A.*?Z/ , then the Z character will be found immediately.
A non-greedy quantifier is also called a minimizing quantifier, since it tends to capture the minimal possible amount of characters and tries to add more only if it did not match the string.

Special characters

Backslash should be used before non-alphabetic characters. Alphabetic characters preceded by a backslash designate special characters:

\t | - tab (0x09)
\n | - line feed (0x0a)
\r | - carriage return(0x0d)
\f | - form feed (0x0c)
\a | - bell (0x07)
\e | - escape (0x1b)
\xNN | - a character in hexadecimal notation, where N belongs to the set [0-9A-Fa-f].
\Q | - beginning of metacharacter quotation
\E | - end of metacharacter quotation
\w | - word character [0-9a-z_A-Z]
\W | - not \w
\s | - whitespace character [ \t\n\r\f]
\S | - not \s
\d | - digit [0-9]
\D | - not \d
\i | - letter [a-zA-Z]
\I | - not \i
\l | - lowercase character [a-z]
\L | - not a lowercase character
\u | - uppercase character [A-Z]
\U | - not an uppercase character
\b | - null token matching a word boundary
\B | - not \b
\A | - text beginning
\Z | - text end
\NN | - backreference to previously matched group in round brackets. NN - integer.

\b means that a character which is a part of a word (\w) is located on the right or on the left of the current position, with a spacer/separator character (\W) on the opposite side.

Apart from grouping, round brackets are used for the following operations:

(?:pattern) | - same as regular grouping, except for this match won't be saved and thus obtains no reference number (N).
(?=pattern) | - zero-width positive lookahead assertion. For example, \w+(?=\s) matches a word followed by whitespace, without including the whitespace in the MatchResult.
(?!pattern) | - zero-width negative lookahead assertion. For example foo(?!bar) matches any occurrence of "foo" that isn't followed by "bar". Remember that this is a zero-width assertion, which means that a(?!b)d will match ad because a is followed by a character that is not b (the d) and a d follows the zero-width assertion.
(?<=pattern) | - zero-width positive backward assertion. The pattern should be of fixed length, i.e. quantifiers cannot be used.
(?<!pattern) | - zero-width negative backward assertion.

Backreferences

Each pair of round brackets (except for the cases listed above) has a unique sequential number - according to the order of appearance of the opening brackets.

\N - where N stands for the number of the bracket - matches the text fragment matched previously by the respective bracket.

Example: /(['"])hello\1/ matches both "hello" or 'helo'.

Options

i | - ignore case
s | - process all input as single-line text. '.' matches any character.
m | - process all input as multi-line text. ^ and $ match respectively any beginning/end of any "internal" line.
x | - spacer. Ignored if not preceded by a backslash. Useful for structuring complex expressions.


Redefining test script settings in sequences

When a HTTP request is listed in any of the sequences (InitSeq, FinalSeq, TestSeq), both the request sending parameters and the values of its local parameters can be set or redefined.

The syntax is as follows:

[TestSeq]
Request1 (param1=value, param2=value2, $uri=servlet)

Example:

Launching the following test script:

[General]
Method=GET
Server=myserver
Port=5000
URI=/servlet

[Request1]
param1=value1
param2=value2

will result in sending the following HTTP request:

GET /servlet?param1=value1&param2=value2 HTTP/1.1
Host: myserver:5000

Now let us redefine several parameters when calling Request1, and we will obtain the following example:

[General]
Method=GET
Server=myserver
Port=5000
URI=/servlet

[TestSeq]
Request1 (param1=value3, $URI=/servlet/view, port=5555)

[Request1]
param1=value1
param2=value2

Launching this script will result in sending the following HTTP request:

GET /servlet/view?param1=value3&param2=value2 HTTP/1.1
Host: myserver:5555

You can modify all request parameters (except for $exportvar and $fail) by setting request parameters within a sequence.


Test script examples

Performance measurements

To ensure correctness and sufficient precision of the measurements, the testing procedure should meet the following requirements:

  • The CPU load of the computer running the tests should not exceed 90% (during test run).
  • If the amount of virtual clients is greater than 1, the start and end points of the measurement period should be located within the time period when ALL virtual clients are active (see the description of delaystart, maxrequests).

    Recommended settings for the [General] section:

[General]

delaystart=0

Provides simultaneous start of virtual clients (see description of delaystart)

maxrequests=50000

The maxrequests value should not exceed the product (clients x requests). Required to eliminate the “final steps” (see the description of maxrequests)

nowait=0

Do not use the timeout check. Allows to provide significant server load using a single workstation, thanks to saved resources. Consequently, can lead to hang-ups of virtual clients in case of a big amount of stuck requests (which would be normally closed using the timeout check)..

log=0

Logging is off - to reduce resource consumption of the workstation running the test.

nodetails=1

Turns off the printing of detailed statistics on separate requests for all virtual clients. Useful if measuring only the value of Requests per second.

summarytimes=1

Turns on printing of the summary statistics

Example: a test script for measuring the performance of loading the main page of a Web server:

[General]
clients=50
requests=10000
server=myserver
keepalive=1
delaystart=0
maxrequests=50000
nowait=0
log=0
nodetails=1
summarytimes=1

[TestSeq]
Req1

[Req1]
$uri=/


Using external scripts

The support of scripting in Web Roller allows to replace the sequence of requests with a program.

Web Roller scripting is implemented via the ActiveScriptingHost, i.e. any scripting language using the respective interface can be used. Currently JScript, VBScript and PerlScript (the latter requires installation of ActiveState ActivePerl) are supported. The name of the script file is set by the Script parameter of the [General] section. If this parameter is specified, the scripting is turned on automatically. It is also possible to include the script text in the body of a test script. In this case $inline.ext should be used as the name of script file.

Caution!: It is important to specify a correct extension - it is used to determine the script language:

  • Perl - .pl
  • Jscript - .js
  • VBScript - .vbs

Scripting imposes certain limitations:

  • [InitSeq], [FinalSeq] and [TestSeq] sequences are ignored;
  • $Seq and $List functions cannot be used;
  • The following parameters are ignored: requests
      (to use requests you will need to set up a cycle - see Example
      2 in GetOpt); timestamp;
      maxtime; abortseq;
      nodetails; maxrequests;
      exacttiminginterval;
      ignoreerrors;

The rest of the settings and functions described earlier remain functional. Web Roller transfers the control to the script program and does not issue any requests itself.

To provide operation with HTTP requests the wapt object is added to the script's namespace, offering the following methods:

AddRequest

int AddRequest(string RequestName, string Uri);

Declare a new HTTP request with name RequestName. Returns 1 upon success, 0 if a reauest with this name already exists.

Example:

$wapt->AddRequest("test","/");
$wapt->AddRequest("test","http://myserver");

AddParam

int AddParam(string ReqãuestName, string ParamName, string ParamValue);

Adds the Param=Value pair in the RequestName request.

Example:

$wapt->AddParam("test","param1","value4");

SetParam

int SetParam (string RequestName, string Param[, string Value]);

Sets the value of existing Param parameter in the existing RequestName request. If Value is not specified, then the parameter will be reset to the original value specified in the request declaration. Returns 1 upon success, otherwise 0.

Example:

$wapt->SetParam("test","param1","value9");

ResetRequest

int ResetRequest (string RequestName)

Restores the initial request parameters values modified by SetParam, AddParam for the RequestName request.

Example:

$wapt->AddParam("test","param1","value4");
$wapt->SetParam("test","param1","value9");
$wapt->ResetRequest("test");

Request

int Request (string RequestName);

Issue a RequestName request. Returns 0 if the request does not exist (or unsuccessful), 1 upon successful request termination.

Example:

$wapt->Request("test");

SetRequestOpt

int SetRequestOpt(string RequestName, string Options);

Sets the parameters ($uri, $server, $port, $delay, $method, $keepalive) for the RequestName request. The Options string should be in the same format as when redefining parameters from within a sequence.

Note: If Perl is used, the Options string should be in single quotes:

$wapt->SetRequestOpt("test",’$uri=/view’);

GetOpt

string GetOpt (string Option);

returns the value of the Option parameter from the [General] section. Additionallly returns clid – the ordinal number of the current virtual client.

Example:

$wapt->GetOpt("requests");

Response

string Response ();

Returns the response to the last HTTP request.

IsAborting

int IsAborting ();

Returns 1 if Ctrl-Break was pressed, otherwise 0. If you want the script to properly terminate on pressing, this value should be checked frequently.

Logit

void Logit (string logstr);

Writes the logstr string to the logfile, if logging is on ([General] section, Log=1).

Examples

Example1. Primitive Perl script:

[General]
script=$inline.pl

[script]
$wapt->AddRequest("test","http://myserver/");
$wapt->Request("test");
[/script]

This will result in sending the following HTTP request to the server:

GET / HTTP/1.1
Host: myserver

Example2. Using the Requests parameter

[General]
log=1
script=$inline.pl
requests=3

[script]
$wapt->AddRequest("test","http://myserver/");
for($i=0;$i<$wapt->GetOpt("requests");$i++)
{
$wapt->Request("test");
}
[/script]


Test run parameters

The parameters of a test run can be set either by editing the source of a test project file , or from the Run Dialog in the Run Tab.

The syntax of a test project (.WPJ file) is similar to that of .INI files - it consists of sections (section names in square brackets). General syntax of settings used in the file is <name>=<value>, with section names in square brackets. You can start a line with a semicolon character to write comments or to temporarily disable some lines. Empty lines are ignored and can be used for formatting.

Each test project should contain:

  • [Scripts] section - listing all test scripts (.TST files) that should be included in the current test project - use fully qualified pathnames; one file per line.
  • [Runoptions] section - listing which test scripts to run and which parameters of the scripts should be modified for the current script run; one script run per line.

Use the the following format for script runs:

Full_Path_to _TST_File|"parameter_1=value_1","parameter_2=value_2", . . . "parameter_N=value_N"

To set a fixed value of the parameter for the current execution, specify a static textual or numeric value_X.

To make a test script run multiple times with a numeric parameter cycling from start_value to end_value with the specified step, declare a Range in the format [start_value..end_value:step]. If the step value is 1, the colon (:) and step can be omitted.

Examples:

[Scripts]
c:\tests\webroller\current\lasttest.tst
[Runoptions]
c:\tests\webroller\current\lasttest.tst|"clients=20","requests=50"

This will override the original .TST values for clients and requests, setting them respectively to 20 and 50.

[Scripts]
c:\tests\webroller\current\lasttest.tst
[Runoptions]
c:\tests\webroller\current\lasttest.tst|"clients=[10..50:10]"

This will override the original .TST value for clients and run the script 5 times, setting clients sequentially to 10, 20, 30, 40 and 50.

[Scripts]
c:\tests\webroller\current\lasttest.tst
[Runoptions]
c:\tests\webroller\current\lasttest.tst|"clients=[1..5]"

This will override the original .TST value for clients and run the script 5 times, setting clients sequentially to 1, 2, 3, 4 and 5.

The Run dialog allows to modify the settings of a single script run in the project (see also the Glossary). The following controls are available:

  • Select script combobox: a list of full paths to script files included in the current project. Here you select which test script to run and modify.
  • Select parameter combobox: a list of all parameters from the [General] section of the script.
  • Modifiers. You can set any of the script's global parameters to a specific value (other than initially specified in the script), or make numeric parameters cycle from one value to another with a desired step:
    • Select set to from the combobox and enter a new value for a desired parameter, or:
    • Select change from and enter the start value, end value and the step to make the selected parameter cycle through these values with the desired step.
  • Redefined parameters area: displays the set of modifiers for the current script run (in the order they will be saved in the test project file). The contents of this area is, actually, an alternative view of a single script run (hwich is equivalent to a single line from the [runoptions] section of the test project file). Each line in the window stands for a single modifier. If the same parameter was redefined multiple times (i.e., it occurs in several lines), the latest (lowest) setting will override the previous ones.
    See Test Projects for details on modifiers syntax and structure of test project (.WPJ) files.

- add a new parameter (with the currently set modifiers) to the current script run;

- edit the highlighted parameter (or right-click the desired line and select Edit);

- delete a parameter from the script run (or right-click the desired line and select Delete).

- confirm changes and exit the Run dialog;

- discard changes and exit the Run dialog.


Running from Win32 console

The wr.exe executable can be used to launch test scripts and projects in console mode (from the command line). To launch a test project enter:

wr.exe projectname.wpj

To launch a single test script enter:

wr.exe testscriptname.tst [-option=value,., , ]

[Options] can be set if the $gvar function is used within the test script.

Example:

The server parameter in the test script testscript.tst is set as follows:

Server=$gvar(server, myserver)

If we launch the script this way:

wr.exe testscript.tst

the requests will be sent to myserver. But if we launch the script as:

wr.exe testscript.tst -server=testserver

then the requests will be sent to testserver.


Testing results and assessment

After the execution of a test script the textual and graphical execution reports are created.

Execution report

The textual report is generated as a file with .xls extension saved in the current folder; the concise version of the report is also presented on the Report->Report tab.

Let us examine a typical execution report. Henceforward all comments and explanations are in bold:

Report generated by Web Roller v3.5 build 939 |  |  |  |  |
Test started at:Sun Mar 31 20:41:26 2002 |  |  |  |  |
Test finished at:Sun Mar 31 20:41:26 2002 |  |  |  |  |
Project name: Test project |  |  | Project name (Projectname setting)
Description: Test |  |  | Project desctription (Description parameter)
Server name: kort |  |  | Server tested (Server parameter)
Clients: 3 |  |  | Number of virtual clients
Requests: 5 |  |  | Number of [TestSeq] runs
Test executed by: Igor |  |  | User name
Test executed at: kort |  |  | Workstation name
Clnt/Rqst Req1 Req2 Date and time of request execution (if timestamp=1)
C00R00000 5.07 51.06 2002/03/31 20:41:26 2002/03/31 20:41:26 |  |  |  |  | C00R00000 – Client0, Request0. First run of [TestSeq] by virtual client #1, processing times for each request from [TestSeq] are listed in milliseconds.
C00R00001 0.22 125.02 2002/03/31 20:41:26 2002/03/31 20:41:26
C00R00002 61.18 89.67 2002/03/31 20:41:26 2002/03/31 20:41:26
C00R00003 42.41 112.37 2002/03/31 20:41:26 2002/03/31 20:41:26
C00R00004 6.81 69.35 2002/03/31 20:41:26 2002/03/31 20:41:26
C01R00000 4.57 89.27 2002/03/31 20:41:26 2002/03/31 20:41:26
C01R00001 32.19 90.65 2002/03/31 20:41:26 2002/03/31 20:41:26
C01R00002 22.72 79.29 2002/03/31 20:41:26 2002/03/31 20:41:26
C01R00003 41.30 114.60 2002/03/31 20:41:26 2002/03/31 20:41:26
C01R00004 26.73 81.37 2002/03/31 20:41:26 2002/03/31 20:41:26
C02R00000 16.14 68.92 2002/03/31 20:41:26 2002/03/31 20:41:26
C02R00001 6.52 80.96 2002/03/31 20:41:26 2002/03/31 20:41:26
C02R00002 29.99 99.36 2002/03/31 20:41:26 2002/03/31 20:41:26
C02R00003 26.81 61.01 2002/03/31 20:41:26 2002/03/31 20:41:26
C02R00004 18.99 62.73 2002/03/31 20:41:26 2002/03/31 20:41:26
--- Basic statistics --- | Minimum, average and maximum processing times for each of HTTP requests (Req1, Req2) - milliseconds.
Min: 0.22 51.06
Avg: 22.78 85.04
Max: 61.18 125.02
--- Network traffic details --- |  |  |  |  |
Total bytes sent : 2604 | Total amount of data sent and received.
Total bytes received : 25320
--- Summary times --- |  |  |  |  |
Virtual Clients statistics: | Virtual clients statistics: Count – amount of HTTP requests sent; Time – virtual client operation time; AR/S – average amount of HTTP requests per second; AT/R – average time of HTTP request processing.
Count Time AR/S AT/R
10 601 16.64 60.10
10 601 16.64 60.10
10 500 20.00 50.00
Total work time: 631 | Script operation time
Total requests made: 30 | Total amount of HTTP requests made by all virtual clients
Total average time per request: 21.0333 |  |  | Average HTTP request processing time (using all requests)
Total average requests per second: 47.5436 |  |  | Average amount of HTTP requests per second
WARNING: approximation used! |  |  |  | The warning is displayed if the overall time of the test script operation is lower than required for precise measurement (and calculation) of the “Total average time per request “ and “Total average requests per second” values. In this case the results will be approximated.
Not enough requests per client for exact timing!
--- HTTP response codes details --- |  | Statistics on HTTP response codes: Code – response code; Count – amount of responses with this code.
Code Count
200 30

Execution report (if scripting is used)

If scripting is used, the textual execution report is simplified. Execution of the script below:

[General]
project=Test Project
Description=test
script=$inline.pl
requests=3
clients=2

[script]
$wapt->AddRequest("test1","http://myserver/");
$wapt->AddRequest("test2","http://myserver/");
for($i=0;$i<$wapt->GetOpt("requests");$i++){
$wapt->Request("test1");
$wapt->Request("test2");
}
[/script]

will result in the following report:

Report generated by Web Roller v3.5 build 957 |
Test started at:Fri Apr 19 16:28:42 2002 |
Test finished at:Fri Apr 19 16:28:43 2002 |
Project name: Test Project | Project name (Projectname setting)
Description: Test | Project desctription (Description parameter)
Server name: myserver | Server tested (Server parameter)
Clients: 2 | Number of virtual clients
Requests: 3 | Number of [TestSeq] runs
Test executed by: Igor | Computer user name
Test executed at: Kort | Computer workstation name
--- Client 0 --- | Statistics on processed requests for virtual client “Client 0”
test1 09.20
test2 12.10
test1 11.60
test2 09.70
test1 12.10
test2 11.60
--- Client 1 --- | Statistics on processed requests for virtual client “Client 1”
test1 09.20
test2 12.10
test1 11.60
test2 09.70
test1 12.10
test2 11.60
--- HTTP response codes details --- | Statistics on HTTP response codes: Code – response code; Count – amount of responses with this code.
Code Count
200 12

Execution Log

If in the [General] section Log=1, all the results of operation are logged to a file. Each virtual client generates a separate log file. The complete log of the Web Roller client operation is written to this file. This includes all requests, responses, redirects and the HTML code returned.

A test script like this:

[General]
clients=1
requests=2
server=kort
log=1

[TestSeq]
Req1
Req2

[Req1]
$uri=/
$method=GET

[Req2]
$uri=/view.pl
$method=GET

will result in the following log file:

----- [TestSeq]: Req1(0) ----- | Separator line: pass #0 of [TestSeq], request Req1
Request: | Header of the HTTP request sent
GET / HTTP/1.1
Host: kort
Connection: close
Response: | Header of the server response.
HTTP/1.1 200 OK
Date: Sun, 21 Apr 2002 16:30:45 GMT
Server: Apache/1.3.12 (Win32)
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> | HTML code of response
<html>
</body>
..
..
</body>
</html>
----- [TestSeq]: Req2 (0) ----- | Separator line: pass #0 of [TestSeq], request Req2
Request: |
GET /view.pl HTTP/1.1 |
Host: kort |
Referrer: http://kort/ |
Connection: close |
Response: |
HTTP/1.1 200 OK |
Date: Sun, 21 Apr 2002 16:30:45 GMT |
Server: Apache/1.3.12 (Win32) |
Connection: close |
Transfer-Encoding: chunked |
Content-Type: text/plain |
<html> |
</body> |
.. |
.. |
</body> |
</html> |
----- [TestSeq]: Req1(1) ----- | Separator line: pass #1 of [TestSeq], request Req1
Request: |
GET / HTTP/1.1 |
Host: kort |
Referrer: http://kort/view.pl |
Connection: close |
Response: |
HTTP/1.1 200 OK |
Date: Sun, 21 Apr 2002 16:30:45 GMT |
Server: Apache/1.3.12 (Win32) |
Connection: close |
Transfer-Encoding: chunked |
Content-Type: text/html |
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> |
<html> |
</body> |
.. |
.. |
</body> |
</html> |
----- [TestSeq]: Req2(1) ----- | Separator line: pass #1 of [TestSeq], request Req2
Request: |
GET /view.pl HTTP/1.1 |
Host: kort |
Referrer: http://kort/ |
Connection: close |
Response: |
HTTP/1.1 200 OK |
Date: Sun, 21 Apr 2002 16:30:45 GMT |
Server: Apache/1.3.12 (Win32) |
Connection: close |
Transfer-Encoding: chunked |
Content-Type: text/plain |
<html> |
</body> |
.. |
.. |
</body> |
</html> |


Test scripts

The syntax of a test script (.TST file) is similar to that of .INI files - it consists of sections (section names in square brackets). General syntax of most settings used in the file is like this: <name>=<value>. You can start a line with a semicolon character to write comments or to temporarily disable some lines. Empty lines are ignored and can be used for formatting.

Each test should contain a [General] section and a set of requests. Optionally, the sections [InitSeq], [TestSeq], [FinalSeq], [Includes] and [Headers] can be defined.

  • [General] section defines the global test parameters, default values for requests and various settings for controlling the behavior of virtual clients and the output files format.
  • [InitSeq] section defines the sequence of requests to be run once before the main [TestSeq] sequence.
  • [TestSeq] section defines the sequence of requests for the main testing cycle.
  • [FinalSeq] section defines the sequence of request to be run once after completion of [TestSeq].
  • [Includes] section lists external .TST files to be used within the current test. If you use external tests, the settings from their [General] section can override the settings of your file, but only if the [Includes] section is located below your [General] section. The same is true for [InitSeq], [TestSeq] and [FinalSeq] sections.
  • [Headers] section allows to define additional fields in the HTTP request header (UserAgent, Referrer etc.).

[InitSeq] and [FinalSeq] are considered empty by default. If the [TestSeq] is not explicitly declared, then all the requests defined in the file are considered to compose the [TestSeq] (in the order they appear in the file). These three test sequences are composed of request names, one per line.

You can redefine settings of a specific request within a sequence. List modified settings (using the <name>=<value> pairs) in round brackets immediately after the request name, separating pairs with commas. If <value> contains a comma inside, enclose the whole <name>=<value> pair in double-quotes (“).

Example:

[TestSeq]
request1
request1($method=get,"static=this,is+a+test",combined=yahoo!)

You can as well redefine certain global request parameters - using, e.g., $uri, $delay etc.

Example:

[request1]
$uri=/scripts/test/params.pl
$method=post
$port=8001

You can concatenate multiple functions to compose a single request parameter using the & character.

Example:

CombinedParam=Hello&$Nrand(10,99)&$Arand(5,10)


Syntax - global parameters

Syntax of global parameters

Parameter | Type | Default value | Description

Globals:
Clients | Integer | 1 | Amount of virtual clients.
Requests | Integer | 50 | How many times the main Test sequence [TestSeq] will be executed.
Server | String (host) | Localhost | Name (or IP) of the server being tested.
Uri String / Default URI (can be redefined in specific requests).
Port | Integer | 80/443 | Port number of the tested application. Defaults to 80 for HTTP and to 443 for HTTPS (secure connections).
Method | GET/POST | GET | Default HTTP request method (can be redefined in specific requests).
Timeout | Integer | 30 | Maximal time that Web Roller will wait for a response to a single request before dropping the connection (seconds).
Delay | Integer | 0 | Delay inserted after each subsequent request (milliseconds).
Keepalive | Yes/No | No | Triggers the use of keep-alive requests.
Secure | Yes/No | No | Triggers the use of HTTPS (secure connections).
Httpver | 1.1 or 1 | 1 | Version of HTTP protocol used by virtual clients.
Delaystart | Integer | 0 | Delay between the start of subsequent virtual clients. By default, the server load with a great amount of virtual clients will rise quickly, but not abruptly. If we set this parameter to zero, all virtual clients will be launched exactly at the same time.
Softdelay | Yes/No | No | Triggers a special mode of using the Delay: instead of pure Delay, the value of (Delay - total_request_time) will be used. This mode allows to provide a more even server load.
Proxy | String (host:port) | <empty> | Name (or IP) and port of the external proxy server.
Proxynocache | Yes/No | No | Triggers use of the external proxy caching.
Maxrequests | Integer | 0 | Sets the limit for total amount of requests issued by all virtual clients. If this limit is reached, all clients will be shut down (in no dependence with the amount of requests issued by each separate client ). Zero = disabled.
Nowait | Yes/No | No | Disable the timeout check. Allows to provide massive server loads using a single client computer. Consequently, can lead to hang-up of individual virtual clients.

Scripting:
Scriptfile | N/A | Either a fully qualified filename of the file with script (must reside on local filesystem), or specify the language of inline script by using either of $inline.js, $inline.pl or $inline.vbs). Currently JScript, PerlScript and VBScript are supported (see also Scripting).
Endofscript | [/script] | The sequence of characters indicating the end of a script.

Runtime:
Proxyemulation | Yes/No | No | Adds the fields emulating access of multiple clients via a proxy (X-Forwarded-For) to the HTTP requests. Useful for emulating user activities from different IPs'.
Proxymask String (IP address) 192.168.$C2.$C1 IP mask used by proxyemulation. Allowed meta-numbers:
$C1 - low byte of the virtual client number.
$C2 - high byte of the virtual client number.
$R1 - low byte of the request number.
$R2 - high byte of the request number.
All values span from 1 to 254.
To emulate a new IP for each request, you can set the mask as follows:
$C2.$C1.$R2.$R1
If the amount of requests exceeds 64262, the numbers of $Rx will obviously be repeated.
You can use a static number instead of the default $C2 if the number of clients does not exceed 253.
Maxtime | Integer | 0 | The maximal time for the test run (seconds). A Test script will run until all requests are completed or its running time exceeds Maxtime. Zero = disabled.
Abortseq | Yes/No | No | Selects whether to abort the running sequence if Maxtimeis reached or to wait for normal sequence termination.
Noredir | Yes/No | No | Triggers processing of redirects. By default, Web Roller processes redirects - i.e. if the response to some request contains a redirect, Web Roller will make a respective request. The total of all redirects for the initial request will be accounted as the response time. If a redirect results in an error, the whole request will be accounted as failed.
Ignoreerrors | Yes/No | No | Triggers whether to abort the sequence upon obtaining errors. Useful when running “negative” tests, when an error is the expected result.
MaxSpeed | Integer | 0 | Allows to set network connection bandwidth limit for each virtual client. Useful for emulating modem connections. Set in kbps (kilobits per second). Zero = no limit.
TimingMode Integer 0

Sets the mode of Web Roller time measurements.
0 - WebTransaction: time between the last request byte sent and the last response byte received;
1 - ResponseTime: server response time (only if HTTP1.1 is enabled in HTTPver). Time between the last request byte sent and the first response byte received;
3 - RequestTime: total time of request processing (between the firs sent reauest byte and the last received responce byte).
Authorization | String (login:password) | <empty> | Basic authorization credentials. Substitutes the specified login:password for requests requiring basic authorization.

Output:
Outfile String %n-%s-%cx%r.xls The filename to write the summary results and statistics to. The resulting Tab-separated files are suitable for immediate work with MS Excel.
Meta-characters:
%n - project name;
%s - server name (see server above);
%c - virtual client number;
%r - amount of requests per client (see requests above).
Log | Yes/No | No | Triggers whether to write detailed log files (including all requests respective server responses). A separate log file is created for each virtual client.
Showbadresults | Yes/No | Yes | Triggers whether to list the requests which resulted in errors to the summary file.
Httperrorsymbol | String(8) | HTTPERR | A string indicating HTTP protocol errors - 400, 500, 404, 403 etc. Written to the summary file instead of response times.
Socketerrorsymbol | String(8) | NETERR | A string indicating network (socket) errors. Written to the summary file instead of response times.
Failerrorsymbol | String(8) | FAIL | A string indicating that the $fail error condition was obtained. Written to the summary file instead of response times.
Timeouterrorsymbol | String(8) | T/O | A string indicating that a request timeout occurred. Written to the summary file instead of response times.
Reportinfo | Yes/No | No | Triggers whether to include detailed test info into the summary file (amount of clients and requests, the username and workstation name of the client computer, projectname and description if set).
Projectname | String | <empty> | Project name used by reportinfo.
Description | String | <empty> | Test description used by reportinfo.
Round | Integer | 10 | Sets the rounding limits. Zero = no rounding (minimum detected value = 0.01 mS).
Timestamp | Yes/No | No | Triggers writing the times of starting and finishing of each test sequence to the summary file.
Sumredirecttime | Yes/No | Yes | Triggers totaling of request times for redirects.
Nodetails | Yes/No | No | Switch on/off writing of details for individual requests in the summary file.
Printdate | Yes/No | No | Triggers whether to print the test date/time to the summary file.
Appendresult | Yes/No | No | Triggers whether to append results (logs, summaries) to existing files with same names, or to erase existing files and start new ones.
Summarytimes | Yes/No | Yes | Triggers printing of overall statistics in the summary file.
Exacttiminginterval | Integer | 1000 | Minimum time (in milliseconds) required for accurate estimation of summarytimes. During this period ALL virtual clients should be active.
Locale | String | <system default> | International settings for results output (date, time and delimiters format). To obtain a comma (“,”) as the decimal delimiter, set one of the appropriate locales (e.g. Russian).


Syntax - request parameters

Syntax of request parameters

Parameter | Type / syntax | Description
$server= | String | Name (or IP) of the server being tested.
$port= | Integer | Port number of the tested application.
$uri= | URI or: $href(text) $action $var(name,default) | URI of the current request. Either static text or one of the following expressions: $href(text) - extracts URI from the link containing the specified text (in the response to the previous request). $action - extracts URI from the form action (in the response to the previous request). $var(name,default) - uses the value of the variable set previously by the $exportvar function (see below). If the variable is undefined, uses default instead.
$method= | GET/POST | HTTP request method.
$keepalive= | Yes/No | Triggers the use of keep-alive requests.
$delay= | Integer | Delay inserted after request (milliseconds).
$fail= | [!]/regexp/ | If the regular expression is matched in the response, the request is accounted as failed (see Failerrorsymbol in Syntax - global parameters). The exclamation point (!) preceding the regular expression inverts the matching result: i.e., a request is accounted as failed if the regular expression was NOT matched.
$exportvar= | {varname},/regexp/,{replacestring} | Extracts the value (from the response to previous request) according to regexp and saves the respective replacestring to the external variable varname. Replacestring can use the $0-$9 constructs - according to blocks matched by the regular expression. $0 stands for the whole matching expression.


Syntax - request functions

Syntax of request functions

Function | Description
$NRand({from},{to}[,{prefix}]) | A random integer in the range [from,to], with an optional {prefix} in the beginning.
$ARand({from},{to}) | Random sequence of alphanumeric characters (length limited by from and to).
$LRand({value1},[{value2},…]) | A random value from the list.
$Seq({prefix}[,{startvalue}[,{clientshift}]]) | Sequence of integers increasing by 1 with each subsequent request. Starts from {startvalue} (optional), otherwise from zero. {clientshift} (optional) shifts the start value for different virtual clients. You can add several leading zeroes to startvalue so that the resulting value will always consist from the same number of digits.
$UrlParam[({name})] | The response to the previous request will be searched to find a GET request with {name} as one of parameters. The value of this parameter will be returned; if {name} is omitted, the name of the target parameter will be used instead.
$Hidden[({name})] | Searches the response to previous request for a hidden construction like <.... name=”{name}” value=”{value}”> Returns {value}.
$RegExp(/{regexp}/,{replacestring}) | Performs a match of the result of previous request against {regexp}. Returns {replacestring}. $0-$9 can be used in {replacestring} to substitute blocks of {regexp} delimited by parentheses. $0 stays for the whole match.
$File({filename}[,{mimetype}]) | File transfer (upload) using multipart form data. If no {mimetype} specified, application/octet-stream will be used.
$rxname(/{regexp}/,{replacestring},{value}) | Modifying the parameter name using a regular expression. It does not matter what name is specified in the file: it will be replaced with the one derived from {replacestring}. Use the same rules as for $regexp(). {value} can be any correct value used for request parameters.
$var({name}[,{default}) | Use the variable previously exported by $exportvar. The {default} value is used if the requested variable is undefined.
$gvar(OptionName, DefaultValue) | Defines the parameter value when running as a Win32 console application (see Console mode). OptionName - the name of command line option, DefaultValue - default value of $gvar function. Upon the launch of a test script with a $gvar function the latter substitutes the value of the respective option from the command line (or the default value, if the requested OptionName was not found in the command line). Can be used for defining any parameters (or their components). CANNOT be used to define a parameter name (i.e., cannot stay to the left of the "=" sign).
$clid({prefix}) | {prefix} plus the ordinal number of the virtual client. Useful if you need a value which is different for different clients, but stays unchanged for the same client.
$list(item[,item[,…]]) | This one resembles $LRand, but the elements are substituted in the order they are listed (in a cycle).
$enc<< | The text after << will be URL-encoded


Test projects

The syntax of a test project (.WPJ file) is similar to that of .INI files - it consists of sections (section names in square brackets). General syntax of settings used in the file is <name>=<value>, with section names in square brackets. You can start a line with a semicolon character to write comments or to temporarily disable some lines. Empty lines are ignored and can be used for formatting.

Each test project should contain:

  • [Scripts] section - listing all test scripts (.TST files) that should be included in the current test project - use fully qualified pathnames; one file per line.
  • [Runoptions] section - listing which test scripts to run and which parameters of the scripts should be modified for the current script run; one script run per line.

Use the the following format for script runs:

Full_Path_to _TST_File|"parameter_1=value_1","parameter_2=value_2", . . . "parameter_N=value_N"

To set a fixed value of the parameter for the current execution, specify a static textual or numeric value_X.

To make a test script run multiple times with a numeric parameter cycling from start_value to end_value with the specified step, declare a Range in the format [start_value..end_value:step]. If the step value is 1, the colon (:) and step can be omitted.

Examples:

[Scripts]
c:\tests\webroller\current\lasttest.tst
[Runoptions]
c:\tests\webroller\current\lasttest.tst|"clients=20","requests=50"

This will override the original .TST values for clients and requests, setting them respectively to 20 and 50.

[Scripts]
c:\tests\webroller\current\lasttest.tst
[Runoptions]
c:\tests\webroller\current\lasttest.tst|"clients=[10..50:10]"

This will override the original .TST value for clients and run the script 5 times, setting clients sequentially to 10, 20, 30, 40 and 50.

[Scripts]
c:\tests\webroller\current\lasttest.tst
[Runoptions]
c:\tests\webroller\current\lasttest.tst|"clients=[1..5]"

This will override the original .TST value for clients and run the script 5 times, setting clients sequentially to 1, 2, 3, 4 and 5.


Web Roller - description of parameters and functions

Global settings

The [General] section defines global settings of the test, default values of certain parameters and options controlling the behavior of the virtual clients and the format of reports. These global settings will be used if not overridden by the script run parameters or local HTTP request declarations.

Clients

Syntax example:

clients=5

Sets the amount of virtual clients.

Requests

Syntax example:

requests=10

Sets the number of runs of the main cycle [TestSeq] for each virtual client. E.g. if the settings are Clients=10, Requests=20 and there is a single HTTP request declared in [TestSeq], the overall number of [TestSeq] cycle runs performed by all virtual clients will make 10x20=200, thus 200 HTTP requests will be sent to the server. If there are 5 HTTP requests declared in [TestSeq], then the overall number of [TestSeq] runs in this test will remain the same - 10x20=200, but the amount of HTTP requests sent to the server will make 10x20x5=1000.

Server

Syntax examples:

server=www.webapplicationstesting.com
 server=194.149.225.75

Name or IP address of the server being tested. E.g., www.webapplicationstesting.com or 194.149.225.75

Port

Syntax example:

port=5555

The server port to which the requests are sent.

Secure

If equal to 1, turns on the use of HTTPS. You should understand that when you are testing an application using HTTPS it is impossible to record a test script by capturing the sequence of user actions in the browser. This is due to the fact that requests in this case are encrypted and Web Roller is unable to decrypt them. The test scripts for such a case can be developed using one of two possible alternatives:

During development of test scripts switch the tested application (if possible) to HTTP (instead of HTTPS). Record the sequence of user browser actions using the standard procedure, debug the testing scripts if required. After developing the test scripts, switch the application back to HTTPS and set the Secure option in the test scripts to 1. Develop the scripts manually or within the IDE (refer to Manual - “Manual development of test scripts”).

Timeout

The maximum time of waiting for server response. Set in seconds! If you specify 0, a very big value (ca. 18 hours) will be used, but this will not completely switch off the check. Timeout=1 will most probably result in an error. When the response time exceeds the specified value of Timeout, the request is considered failed and is closed; the report will list "Ò/Î" instead of response time (if other symbols were not specified by the Timeouterrorsymbol setting).

Httpver

The version of HTTP protocol used by the virtual clients.

1 – Version 1.1;
0 – Version 1.0.

Method

Syntax example:

method=GET

The default method of HTTP request (GET or POST) used in the declared requests (see also $method).

Uri

Syntax example:

uri=/categories/category1.html

Default URI (see also $URI).

Keepalive

Determines whether to use keep-alive requests (see also $keepalive):

1 – make all requests keep-alive;
0 – make all requests non-keep-alive.

Delay

Delay after each request (milliseconds). Sets the delay inserted after each of requests within a sequence (InitSeq, FinalSeq, TestSeq).

Delaystart

Delay between starting times of different virtual clients. By default, if the number of virtual clients is big enough, the initial load on the server rises, but not instantly. E.g. if Clients=200, the chart of virtual clients' start times will look as follows,

potentially leading to poorer precision of the measured Requests per second value. If you set Delaystart to 0, all virtual clients will start exactly at the same time:

MaxRequests

Limits the maximum overall amount of requests made by all virtual clients. When this number is reached, all virtual clients will be stopped - disregarding the amount of requests made by each particular client.

This setting can be useful when measuring the precise value of Requests per second. Usually the chart of server load chart with multiple virtual clients looks as follows:

The reason of the stepwise character of the load rise at the initial stage was described earlier (see Delaystart). The reason of the stepwise character of the final stage is similar: it is both the fact that not all virtual clients start at the same time, and the various work times of particular clients (which can differ significantly).

Web Roller calculates the Requests per second value in the following manner:

Requests per second = Total requests made / Working period

where:

Working period - the overall time spent for processing all requests - from the time of sending the very first request till the end of processing the last one;
Total requests made – the overall number of requests made. Calculated as:

Clients x Requests x (Total number of requests in [TestSeq] section)

Thus the Requests per second value is measured incorrectly, since constant load is not provided during the Working period. You cannot provide accurate conclusions about application performance tested under a specific load (200 virtual clients in our case).

The method of avoiding the starting "steps" is described in Delaystart. The MaxRequests setting allows in turn to avoid the final "steps". The required actions are:

  • Set an excessive value of Requests (e.g. 1 000 000);
  • Set the MaxRequests value so that the amount is sufficient for correct calculation of Requests per second (i.e. if we set MaxRequests=10, then the value of Requests per second calculated using only 10 requests can be claimed to have a great calculation error and thus cannot be used as a realistic estimate; if we set MaxRequests=10000, then you can consider the calculation error to be acceptable).

It also a good idea to consider also the running time of the script when setting the MaxRequests value - e.g. if the script successfully performs 10000 requests in 5 seconds, than the Requests per second will be calculated with a noticeable error).

Attention!

You need to remember that Requests sets the amount of runs for the [TestSeq] sequence which can include multiple HTTP requests, while MaxRequests sets the maximum amount of processed HTTP requests for ALL clients. The MaxRequests value should be significantly less than the value of Total requests made (see above for the calculation of this value), calculated for the case when MaxRequests equals to 0 (i.e. not used).

Softdelay

When this value is non-zero, a special Delay mode is used. This is not a value, but a switch! If the Softdelay mode is active, the full time spent on processing the previous request (Previous request response time) is subtracted from the Delay value, yielding the Resulting delay value. The delay equal to the Resulting delay value will be inserted after the next request. The Resulting delay value is calculated dynamically and is not a constant. This mode allows to provide a more even server load, since constant time between sending requests to the server will be maintained in no dependence, of the response times. The approximate value of the time period for sending the requests to the server is set by the Delay value. If the response time is greater than the value of Delay, then the Resulting delay value for the next request will be set to 0.

Rate

This setting was introduced for convenience. This value specifies the desired amount of requests per hour FOR EACH CLIENT! It controls only the frequency of requests. The test must not necessarily run for an hour - it can be 10 minutes, but if it works one hour, the required amount of requests will be made (of course, if the response time is not too slow to keep the specified rate). This command is equivalent to calculating the required Delay value and turning on the SoftDelay mode. Caution! The Delay value should not be changed after setting Rate - otherwise you will obtain unexpected results (the new Delay value will override the one calculated using the Rate setting).

Outfile

The name of the test run textual report file. Since MS Excel™ accepts data in Tab-Separated format used for the resulting report, the generated file has the xls extension, to avoid unneeded renaming.

Outfile can be set both in plain format and using the naming templates. The following macros can be used:

%s - the server
      setting of the General section;
%p - the port setting
      of the General section;
%r - the requests
      setting of the General section;
%c – the clients
      setting of the General section;
%f - the full name of the test script file;
%n - the name of the test script file (no extension);

Current date/time macros:

%y - year;
%m – month;
%d – day;
%t – hhmmss;

%% - the % character.

The default template for the file name is %n-%s-%cx%r.xls

Maxtime

The maximum time for running the test (seconds). The test will run until all requests will be processed, or until the run time exceeds Maxtime.

Caution!!!

For this setting to be active, the amount of requests should be big enough so that all requests are not processed before Maxtime elapses.

Abortseq

Determines whether to abort the current sequence [TestSeq] for all virtual clients if Maxtime is reached, or to wait for normal sequence termination.

Noredir

Handling of redirects:

0 – redirects are processed;
1 - redirects are ignored.

By default Web Roller handles redirects. Thus, if a redirect is obtained in response to a HTTP request, Web Roller will make an additional request to the location specified by the redirect. If yet another redirect is obtained, another request will be made, and so on. By default the total time of processing all redirects in a series is accounted as the response time for the current request. If some of the requests in this chain of redirects results in an error, the request is accounted as failed and the respective error code is written to the test report.

Sumredirecttime

Defines whether to add up request times for redirected requests:

0 – redirect times are not added, only the response time of the very first request is written to the report;
1 - redirect times are added, the total for all redirects is written in the report.

Locale

International settings for the report. E.g. to use “,” as the decimal separator you should set the locale to Russian. The default decimal separator is a period, setting locale to Russian switches to a comma.

Printdate, Reportinfo

Toggle whether to print supplementary test info in the beginning of the report file - including the run date, amount of clients and requests, the user and workstation names of the computer which ran the test. If Projectname and Description are specified, these will be listed as well.

Projectname

Information string. Used mainly for Reportinfo.

Description

Information string. Used mainly for Reportinfo.

Appendresult

Upon launching the test script the results are either appended to the existing report file, or the existing file is replaced with the new one each time the test script is launched.

Summarytimes

Defines whether to print Summary times in the report.

0 – do not print;
1 – print

You should not use Delay or other resource-consuming options (e.g. regular expressions) together with this option, otherwise the results will not be correct. Only the time period when all clients were launched and none of them has already terminated its run should be used for calculation of Summary times. If the amount of requests is not big enough, probably this requirement will not be met and the respective warning about results approximation will be added to the report.

Round

Rounding precision.

10 – round to 0.1;
0 – no rounding, minimal possible value is 0.01 ms.

Log

All the run results are written to the log file. Each virtual client produces a separate log file.

The comprehensive log of the Web Roller client run is created, including all requests, responses, redirects and the returned HTML code. If the response is obtained in chunked transfer encoding, it will be converted to normal presentation.

Proxy

All HTTP requests generated by Web Roller will be made through the specified proxy server. Provide host:port.

Proxyemulation

Toggles the use of additional fields of the HTTP header for imitating different users working through a proxy server. Useful to emulate requests coming from multiple computers.

0 – do not use emulation;
1 – use emulation.

Proxymask

IP address mask used in Proxyemulation:

$C1 – low byte of the client number;
$C2 – high byte of the client number;
$R1 – low byte of the request number;
$R2 - high byte of the request number.

All values span form 1 to 254. To emulate requests coming from unique IP's, set the mask to:

$C2.$C1.$R2.$R1

Obviously if the amount of requests exceeds 64262, the $Rx values will be repeated. You can use a static value instead of $C2 if the amount of clients is lower than 253.

Timestamp

Toggles whether to write the starting and ending times for each request cycle to the report:

0 – do not write;
1 – write.

Httperrorsymbol

A string (up to 8 characters) used in the report instead of response time if a HTTP error was detected (403, 404, 500 etc.). For instance, "HTTPERR".

Socketerrorsymbol

A string (up to 8 characters) used in the report instead of response time if a network (socket) error was detected.

Failerrorsymbol

A string (up to 8 characters) used in the report instead of response time if the $fail error condition was detected.

Timeouterrorsymbol

A string (up to 8 characters) used in the report instead of response time if the response time exceeded the Timeout value.

Showbadresults

Toggles whether to log the requests which lead to errors.

1 – write to the log;
0 – do not write to the log.

NoDetails

Toggles whether to write to the report the time of individual requests or only the summary statistics. It is a good idea to turn off the details when measuring overall performance, i.e. when a precise value of Requests per second is required.

NoWait

Turns off the timeout check. Allows to provide significant server load using a single workstation, thanks to saved resources. Consequently, can lead to hang-ups of virtual clients in case of a big amount of stuck requests (which would be normally closed using the timeout check).

ExactTimingInterval

Sets the value of minimum time period (milliseconds) for simultaneous work of all clients required to correctly estimate SummaryTimes.

IgnoreErrors

Defines whether to abort the sequence if an error was detected for a request. Useful for “negative” tests, i.e. when an error is an expected result.

TimingMode

The mode of time measurement:

0 – WebTransaction, the time period between the last request byte sent and the last response byte received;
1 – ResponseTime, the server response time. Works only under HTTP/1.1. Time period between the last request byte sent and the first response byte received;
2 – RequestTime, full request processing time. Time period between the first request byte sent and the last response byte received.

MaxSpeed

Sets the network connection bandwidth limit for virtual users (in kbps). Useful to emulate e.g. a modem connection.

Authorization

Syntax:

authorization=login:pass

Basic authorization. Substitutes the specified login:pass in requests requiring basic authorization.

Script

Toggles use of test scripting (inline or external). See details in Scripting.

Defining request parameters locally

Each HTTP request declared in the test uses the sending parameters set in the [General] section. But these parameters can be modified when required, and made unique for each request. The essence of these local settings is the same as for the respective [General] section ones. The parameter names are obtained by adding a $ character to the name of the respective parameter from the [General] section ($URI, $Server, $Port and so on). The locally defined parameters have a higher priority then the global ones, i.e. the global values will be used only if no local parameters were set. Examine the following test script:

Example 1

[General]

Method=GET
Server=myserver
Port=5000
URI=/servlet
[Request1]
param1=value1
param2=value2

In this case Request1 will use the values of Server, Port, URI set globally (in the [General] section), since these are not locally redefined. The resulting HTTP request will look as follows:

GET /servlet?param1=value1&param2=value2 HTTP/1.1
Host: myserver:5000

Now let us redefine locally some parameters, and we will obtain

Example 2

[General]

Method=GET
Server=myserver
Port=5000
URI=/servlet

[Request1]

$Server=server2
$Port=8000
$URI=/servlet/view
param1=value1
param2=value2

Here the Request1 will use the locally redefined values - $URI, $Port. The resulting HTTP request will be:

GET /servlet/view?param1=value1&param2=value2 HTTP/1.1
Host: myserver:8000

Not all of the settings from the [General] section can be redefined locally. The parameters that can be redefined are listed below.

$Server

Syntax:

$Server={servername}

Locally defined name (or IP) of the server to send the request to.

$Port

Syntax:

$Port={portnumber}

Locally defined server port to send the request to.

$URI

Syntax:

$URI={uri|$href(text)|$action|$var(name,default)}

Locally defined request URI. Unlike the globally defined one, it can use either a static value or one of the following functions:

$href(text) – searches for the link with the specified text in the HTML code of response to the previous request and takes the URI from this link;

Example:

The HTML code of the response contains:

<A HREF="/manual_3.html">Preparing test scripts</a>

If we use in the subsequent request the $URI=$href(Preparing test scripts) function, we will finally obtain the actual URI of this link, which is equivalent to $URI=/manual_3.html.

$action – the first form in the respnonse to previous request is found and the URI is extracted from the form action.

Example:

The HTML code of the response contains:

<FORM NAME="search" METHOD="GET" ACTION="/cgi-bin/search.pl">
<INPUT TYPE="text" NAME="text" VALUE="" MAXLENGTH=80>
<INPUT TYPE="submit" VALUE="Start search">

If we use in the subsequent request the $URI=$action function, we will finally obtain the actual URI of this form action, which is equivalent to $URI=/cgi-bin/search.pl.

$var(name,default) – the URI value will be taken from the variable previously set by the $exportvar function. If the specified variable is undefined, the default value will be used.

$Method

Syntax:

$Method={GET|POST}

Request method.

$Keepalive

Syntax:

$Keepalive={0|1}

Toggles the use of keep-alive requests.

$Delay

Syntax:

$Delay={number}

Delay (in ms) after request execution.

$Fail

Syntax:

$Fail=[!]/regexp/

If the specified regular expression is matched in the HTML code of the response to the current request, the request will be accounted as failed. (see Failerrorsymbol). If you place the ! character (exclamation point) before the regular expression, the result of matching will be inverted, i.e. the request will be accounted as failed if the response HTML code DOES NOT contain the matching pattern.

Example:

The test script is:

[General]

requests=5
server=www.yahoo.com

[Request1]

$uri=/
$method=GET
$fail=/Yahoo/

The HTML code of the response contains:

<HTML><HEAD><TITLE>Yahoo!</TITLE>

If the request description contains $Fail=/Yahoo/, the request will be accounted as failed and the string specified by the Failerrorsymbol setting will be written to the report instead of request processing time. This setting is useful for functional testing.

$Exportvar

Syntax:

$Exportvar=(varname),/regexp/,{replacestring}

Defines a variable and its value (see also $Var, $RegExp). Extracts the pattern matching the regular expression from the current response HTML code and stores it in a variable named varname. $0-$9 metacharacters can be used in {replacestring} to specify the groups in the text matched by the regular expression.

Functions as request parameters

In this chapter the term "parameters" is used to describe the parameters specified to the right of the “?” sign in the HTTP request string. The functions are used to define the values of HTTP request parameters and provide the ability to dynamically set the required values for the Param=Value pairs. If a function is used, the declaration of a parameter looks like this: Param=$function. The functions substitute the resulting values, with one exception being the $rxname function (see below).

Now let us examine the available functions and the resulting values that can be generated.

$NRand

Syntax:

ParamName=$NRand({from},{to}[,{prefix}])

A random integer from the [from,to] range; precede by a {prefix} (if one specified).

Example:

The test script is:

[General]
clients=1
requests=3
server=myserver
uri=/view

[Request1]

param=$NRand(1,20,ID)

If you run this script , three HTTP requests will be sent to the server:

http://myserver/view?param=ID5
http://myserver/view?param=ID12
http://myserver/view?param=ID2

The part of the param value after ID is generated randomly. If there is nothing specified instead of ID in the script, i.e. the script is as follows:

[General]

clients=1
requests=3
server=myserver
uri=/view

[Request1]
param=$NRand(1,20)

then three different requests will be sent to the server upon the script run:

http://myserver/view?param=5
http://myserver/view?param=12
http://myserver/view?param=2

$ARand

Syntax:

ParamName=$ARand({from},{to})

Random alphanumeric sequence with length in the range [from, to].

Example:

Launching the test script below:

[General]
clients=1
requests=3
server=myserver
uri=/view
.. ..

[Request1]

param=$ARand(1,20)

results in issuing three HTTP requests:

http://myserver/view?param=g
http://myserver/view?param=fndy5lgs5pck
http://myserver/view?param=xfglfgjl4asd098ljk6c

Here all values of param are randomly generated.

$LRand

Syntax:

ParamName=$LRand(value1[,{value2},…])

Random value from the list of values {value}.

Example:

Launching this test script:

[General]

clients=1
requests=3
server=myserver
uri=/view
.. ..

[Request1]

param=$LRand(test4,bobby,alpha)

Results in sending three HTTP requests to the server:

http://myserver/view?param=alpha
http://myserver/view?param=test4
http://myserver/view?param=bobby

$List

Syntax:

ParamName=$List(value1[,{value2},…])

Similar to $LRand, but the elements are used in the order they are listed (not randomly). The 1st request uses the 1st value from the list, the 2nd one uses the second value and so on. After reaching the last value the list is looped, starting again from the first listed value.

$Seq

Syntax:

ParamName=$Seq({prefix}[,{startvalue}[,{clientshift}]])

Augmenting sequence. Increases by 1 with each run of [TestSeq]. The initial value is {startvalue} (if specified, otherwize 0). If the {clientshift} is specified, then the starting value for different virtual clients will be shifted using this value. You can also specify several leading zeroes for the startvalue, so that the numeric component always keeps the same length.

Example 1:

The following test script:

[General]
clients=1
requests=3
server=myserver
uri=/view
.. ..

[Request1]

param=$Seq(test,2)

Results in sending three HTTP requests to the server:

http://myserver/view?param=test2
http://myserver/view?param=test3
http://myserver/view?param=test4

Example 2:

If the following test script is launched:

[General]

clients=3
requests=3
server=myserver
uri=/view
.. ..

[Request1]

param=$Seq(test,2,5)

the virtual client #1 will send three HTTP requests to the server:

http://myserver/view?param=test2
http://myserver/view?param=test3
http://myserver/view?param=test4

virtual client #2 will send three different HTTP requests:

http://myserver/view?param=test7
http://myserver/view?param=test8
http://myserver/view?param=test9

and virtual client #3, again, three different HTTP requests:

http://myserver/view?param=test12
http://myserver/view?param=test13
http://myserver/view?param=test14

$UrlParam

Syntax:

ParamName=$UrlParam[({name})]

The response to the previous request will be searched for a GET request containing a parameter with name {name}. The value of this parameter will be substituted. If {name} is not explicitly set, then the name of the target parameter (ParamName) will be used - i.e., Myparam=$UrlParam is completely equivalent to Myparam=$UrlParam(Myparam).

Example 1:

The HTML code of the response contains:

<A HREF=”http://myserver/view?SessionID=FCHHAB02DF&param=test111”</a>

Use of Myparam=$UrlParam(SessionID) in the next request will result in the actual value of Myparam=FCHHAB02DF.
This function is very useful when you need to use the parameter values generated by the server on-the-fly. This relates, e.g., to various Session variables.

Example 2:

You need to automate a simple scenario of navigating through a certain site using the respective HTTP requests:

1.User registration:

http://myserver/register?User=Alex&Pass=qwerty (a POST request)

2.After registration the user is redirected to the main category listing the links to all resources of the site. At this stage the SessionID session variable is generated. The user selected the Main category by clicking the respective link:

http://myserver/view?SessionID=FCHHAB02DF&Category=main

3.The user selects a link to a particular resource:

http://myserver/view?SessionID=FCHHAB02DF&Category=main&Item=67

If we record this sequence of HTTP request, it would result in the following test script:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/register
$Method=POST
User=Alex
Pass=qwerty

[Req2]
$Uri=/view
$Method=GET
SessionID=FCHHAB02DF
Category=main

[Req3]
$Uri=/view
$Method=GET
SessionID=FCHHAB02DF
Category=main
Item=67

But we will not be able to use this script - after sending Req2 an error will occur. The reason is that we try to use the old value of SessionID. Consequently, we need that each time after user registration the test script operates with a correct (currently generated) value of SessionID. We can modify the script to achieve this:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/register
$Method=POST
User=Alex
Pass=qwerty

[Req2]
$Uri=/view
$Method=GET
SessionID=$UrlParam(SessionID)
Category=main

[Req3]
$Uri=/view
$Method=GET
SessionID=$UrlParam(SessionID)
Category=main
Item=67

and thus solve the problems related to SessionID. Now before sending the Req2 request the $UrlParam function will parse the response (the returned HTML code) to request Req1, find the first link containing the required SessionID and substitute the actual value of SessionID into Req2 (in the Example this value is FCHHAB02DF).

$Hidden

Syntax:

ParamName=$Hidden[({name})]

The function searches for an element containing <.... name=”{name}” value=”{value}”> using the specified {name}. The found {value} will be used. If {name} is not specified explicitly, the name of the target parameter (ParamName) will be used. The syntax of this type is used typically to define input elements in HTML forms. The advantage of this function is that it allows to work e.g. with hidden form elements with dynamically generated values (again the Session variables).

Example:

The response to the previous HTTP request contains a form with a set of hidden elements:

<form action="/cgi-bin/select.pl" method="get">
<input type="text" name="index" size="25" maxlength="25">
<input type="submit" value="Search">
<input type="hidden" name="restrict1" value="yes">
<input type="hidden" name="restrict3" value="no">
....
....
</form>

and the values of these elements should be used in the subsequent HTTP request. Let us compose a test script for solving this problem:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/

[Req2]
$Uri=/cgi-bin/select.pl
$Method=GET
index=AnyText
restrict1=$Hidden
restrict3=$Hidden

In the test script above, before sending the Req2 request the $Hidden function will parse the HTML code of response to request Req1. For the restrict1 parameter the function will find the fragment <….name="restrict1" value=……>, for restrict3 - respectively, <….name="restrict3" value=……>. Respective values will be returned and used as values of parameters restrict1 and restrict3 in Req2.

$RegExp

Syntax:

ParamName=$RegExp(/{regexp}/,{replacestring})

The response to the previous request will be scanned for a match to the specified regular expression {regexp}, and the resulting function value will be built according to the search results and the {replacestring} syntax. The syntax is similar to Perl syntax (see description in Syntax Reference ->Regular Expressions).

Example:

You need to automate the submitting of a form with comboboxes. Only the description of combobox values are static (in this case - minimal, middle and maximum), but the values are generated by the server and not initially known. The HTML code of such a form may look like this:

<form action=”send” method=”post”>
.....
.....
<select name="weight">
<option value="20">minimal</option>
<option value="40">middle</option>
<option value="60">maximum</option>
</select>
</form>

Let us compose a test script for solving this problem:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/

[Req2]
$Uri=/send
$Method=POST
weight=$RegExp(/value=\"(\d)\">minimal</,$1)

Before sending the Req2 request $RegExp parses the HTML code of response to the Req1 request. For the weight parameter it will find the fragment <option value="20">minimal</option>, which matches the regular expression. The value 20 will be extracted and returned by the $RegExp function.

$RxName

Syntax:

$RxName(/{regexp}/,{replacestring},{value})

Modifies the parameter name according to the regular expression. No matter which parameter name was initially used in the script, it will be replaced by the result built according to {replacestring}. The rules are the same as for $regexp(). The {value} can be any correct value used in the request parameters, e.g. the ones listed above.

Example:

You need to submit a form for which the names of input elements are dynamically generated by the server, and thus initially unknown. The form is like follows:

<form action="send">
<input type="checkbox" name="Option124">Food
<input type="checkbox" name="Option78">Juice
<input type="checkbox" name="Option471">Biscuit
</form>

The test script for performing this task looks as follows:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/

[Req2]
$Uri=/send
AnyText=$rxname(/name="(Option\d+)">Juice/i,$1),on

Before sending the Req2 request the $RxName function parses the HTML code returned in response to Req1 and finds the code fragment name="Option78">Juice matching the regular expression. The value Option78 is extracted and used as the parameter name, the value of this parameter is then set to on.

$Var

Syntax:

ParamName=$Var({name}[{,default}])

Uses a variable set earlier by the $exportvar function. If the variable with name {name} is not defined, the {default} value will be used instead (if present).

Example:

Let us examine once again the example for the $RegExp function and solve the problem in a different manner. You need to automate the submitting of a form with comboboxes. Only the description of combobox values are static (in this case - minimal, middle and maximum), but the values are generated by the server and not initially known. The HTML of such a form may look like this:

<form action=”send” method=”post”>
.....
.....
<select name="weight">
<option value="20">minimal</option>
<option value="40">middle</option>
<option value="60">maximum</option>
</select>
</form>

Let us compose a test script for solving this problem:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/
$ExportVar=VarName,/value=\"(\d)\">minimal</,$1

[Req2]
$Uri=/send
$Method=POST
weight=$Var(VarName)

After sending the Req1 request the $ExportVar function parses the HTML code returned in response to Req1 and finds the code fragment <option value="20">minimal</option> which matches the regular expression. The value 20 is extracted and assigned to the VarName variable. Before sending the Req2 request the $Var function returns the value of the VarName variable (defined earlier), and assigns this value to the weight parameter.

All functions parsing the HTML code of the responses have one common limitation - they can operate only with the code of the previous request. Defining and using external variables with $ExportVar and $Var functions allows to overcome this limitation and transfer data between non-subsequent HTTP requests.

$File

Syntax:

$File({path filename}[{,mimetype}])

File upload. Multipart/form-data will be used. If {mimetype} is not explicitly specified, application/octet-stream will be used. Used to upload a file to the server, e.g. when sending e-mail messages with attachments via the Web interface of the mailing system.

Example:

Usually the code for file upload looks as follows:

<form method="post" action="upload" enctype="multipart/form-data">
<input type="file" name="filename">
</form>

If you need to upload the JPEG image image.jpg located in the C:\Graphics\Jpg\ folder, the test script will look as the one below:

[General]
clients=1
requests=1
server=myserver

[Req1]
$Uri=/upload
$Method=POST
filename=$File(C:\Graphics\Jpg\image.jpg,image/pjpeg)

$Clid

Syntax:

ParamName=$Clid({prefix})

{prefix} plus the ordinal number of the virtual client. Can be used to create values which should be different for different clients, but the same for a single client.

$GVar

Syntax:

Paramname=$gvar(OptionName, DefaultValue)

Defines the parameter value when running as a Win32 console application (see Console mode).

OptionName – the name of command line option,
DefaultValue – default value of $gvar function.

Upon the launch of a test script woth a $gvar function the latter substitutes the value of the respective option from the command line (or the default value, if the requested OptionName was not found in the command line). Can be used for defining any parameters (or their components). CANNOT be used to define a parameter name (i.e., cannot stay to the left of the “=” sign).

Examples (correct use):

Server=$gvar(server, myserver)
$Server=$gvar(server, myserver)
param=$gvar(param, value1)
param=abc$gvar(param, value1)

Example (incorrect use):

$gvar(paramname, value)=value9

Passing request parameters from a sequence

When a HTTP request is listed in any of the sequences (InitSeq, FinalSeq, TestSeq), both the request sending parameters and the values of its local parameters can be set or redefined.

The syntax is as follows:

[TestSeq]
Request1 (param1=value, param2=value2, $uri=servlet)

Example:

Launching the following test script:

[General]
Method=GET
Server=myserver
Port=5000
URI=/servlet

[Request1]
param1=value1
param2=value2

will result in sending the following HTTP request:

GET /servlet?param1=value1&param2=value2 HTTP/1.1
Host: myserver:5000

Now let us redefine several parameters when calling Request1, and we will obtain the following example:

[General]
Method=GET
Server=myserver
Port=5000
URI=/servlet

[TestSeq]
Request1 (param1=value3, $URI=/servlet/view, port=5555)

[Request1]
param1=value1
param2=value2

Launching this script will result in sending the following HTTP request:

GET /servlet/view?param1=value3&param2=value2 HTTP/1.1
Host: myserver:5555

You can modify all request parameters (except for $exportvar and $fail) by setting request parameters within a sequence.

Scripting

The support of scripting in Web Roller allows to replace the sequence of requests with a program.

Web Roller scripting is implemented via the ActiveScriptingHost, i.e. any scripting language using the respective interface can be used. Currently JScript, VBScript and PerlScript (the latter requires installation of ActiveState ActivePerl) are supported. The name of the script file is set by the Script parameter of the [General] section. If this parameter is specified, the scripting is turned on automatically. It is also possible to include the script text in the body of a test script. In this case $inline.ext should be used as the name of script file.

Caution!: It is important to specify a correct extension - it is used to determine the script language:

  • Perl - .pl
  • Jscript - .js
  • VBScript - .vbs

Scripting imposes certain limitations:

  • [InitSeq], [FinalSeq] and [TestSeq] sequences are ignored;
  • $Seq and $List functions cannot be used;
  • The following parameters are ignored: requests
      (to use requests you will need to set up a cycle - see Example
      2 in GetOpt); timestamp;
      maxtime; abortseq;
      nodetails; maxrequests;
      exacttiminginterval; ignoreerrors;

The rest of the settings and functions described earlier remain functional. Web Roller transfers the control to the script program and does not issue any requests itself.

To provide operation with HTTP requests the wapt object is added to the script's namespace, offering the following methods:

AddRequest

int AddRequest(string RequestName, string Uri);

Declare a new HTTP request with name RequestName. Returns 1 upon success, 0 if a reauest with this name already exists.

Example:

$wapt->AddRequest("test","/");
$wapt->AddRequest("test","http://myserver");

AddParam

int AddParam(string ReqãuestName, string ParamName, string ParamValue);

Adds the Param=Value pair in the RequestName request.

Example:

$wapt->AddParam("test","param1","value4");

SetParam

int SetParam (string RequestName, string Param[, string Value]);

Sets the value of existing Param parameter in the existing RequestName request. If Value is not specified, then the parameter will be reset to the original value specified in the request declaration. Returns 1 upon success, otherwise 0.

Example:

$wapt->SetParam("test","param1","value9");

ResetRequest

int ResetRequest (string RequestName)

Restores the initial request parameters values modified by SetParam, AddParam for the RequestName request.

Example:

$wapt->AddParam("test","param1","value4");
$wapt->SetParam("test","param1","value9");
$wapt->ResetRequest("test");

Request

int Request (string RequestName);

Issue a RequestName request. Returns 0 if the request does not exist (or unsuccessful), 1 upon successful request termination.

Example:

$wapt->Request("test");

SetRequestOpt

int SetRequestOpt(string RequestName, string Options);

Sets the parameters ($uri, $server, $port, $delay, $method, $keepalive) for the RequestName request. The Options string should be in the same format as when redefining parameters from within a sequence.

Note: If Perl is used, the Options string should be in single quotes:

$wapt->SetRequestOpt("test",’$uri=/view’);

GetOpt

string GetOpt (string Option);

returns the value of the Option parameter from the [General] section. Additionallly returns clid – the ordinal number of the current virtual client.

Example:

$wapt->GetOpt("requests");

Response

string Response ();

Returns the response to the last HTTP request.

IsAborting

int IsAborting ();

Returns 1 if Ctrl-Break was pressed, otherwise 0. If you want the script to properly terminate on pressing, this value should be checked frequently.

Logit

void Logit (string logstr);

Writes the logstr string to the logfile, if logging is on ([General] section, Log=1).

Examples

Example1. Primitive Perl script:

[General]
script=$inline.pl

[script]
$wapt->AddRequest("test","http://myserver/");
$wapt->Request("test");
[/script]

This will result in sending the following HTTP request to the server:

GET / HTTP/1.1
Host: myserver

Example2. Using the Requests parameter

[General]
log=1
script=$inline.pl
requests=3

[script]
$wapt->AddRequest("test","http://myserver/");
for($i=0;$i<$wapt->GetOpt("requests");$i++)
{
$wapt->Request("test");
}
[/script]

Testing results and assessment

After the execution of a test script the textual and graphical execution reports are created.

Execution report

The textual report is generated as a file with .xls extension saved in the current folder; the concise version of the report is also presented on the Report->Report tab.

Let us examine a typical execution report. Henceforward all comments and explanations are in bold:

Report generated by Web Roller v3.5 build 939 |  |  |  |  |
Test started at:Sun Mar 31 20:41:26 2002 |  |  |  |  |
Test finished at:Sun Mar 31 20:41:26 2002 |  |  |  |  |
Project name: Test project |  |  | Project name (Projectname setting)
Description: Test |  |  | Project desctription (Description parameter)
Server name: kort |  |  | Server tested (Server parameter)
Clients: 3 |  |  | Number of virtual clients
Requests: 5 |  |  | Number of [TestSeq] runs
Test executed by: Igor |  |  | User name
Test executed at: kort |  |  | Workstation name
Clnt/Rqst Req1 Req2 Date and time of request execution (if timestamp=1)
C00R00000 5.07 51.06 2002/03/31 20:41:26 2002/03/31 20:41:26 |  |  |  |  | C00R00000 – Client0, Request0. First run of [TestSeq] by virtual client #1, processing times for each request from [TestSeq] are listed in milliseconds.
C00R00001 0.22 125.02 2002/03/31 20:41:26 2002/03/31 20:41:26
C00R00002 61.18 89.67 2002/03/31 20:41:26 2002/03/31 20:41:26
C00R00003 42.41 112.37 2002/03/31 20:41:26 2002/03/31 20:41:26
C00R00004 6.81 69.35 2002/03/31 20:41:26 2002/03/31 20:41:26
C01R00000 4.57 89.27 2002/03/31 20:41:26 2002/03/31 20:41:26
C01R00001 32.19 90.65 2002/03/31 20:41:26 2002/03/31 20:41:26
C01R00002 22.72 79.29 2002/03/31 20:41:26 2002/03/31 20:41:26
C01R00003 41.30 114.60 2002/03/31 20:41:26 2002/03/31 20:41:26
C01R00004 26.73 81.37 2002/03/31 20:41:26 2002/03/31 20:41:26
C02R00000 16.14 68.92 2002/03/31 20:41:26 2002/03/31 20:41:26
C02R00001 6.52 80.96 2002/03/31 20:41:26 2002/03/31 20:41:26
C02R00002 29.99 99.36 2002/03/31 20:41:26 2002/03/31 20:41:26
C02R00003 26.81 61.01 2002/03/31 20:41:26 2002/03/31 20:41:26
C02R00004 18.99 62.73 2002/03/31 20:41:26 2002/03/31 20:41:26
--- Basic statistics --- | Minimum, average and maximum processing times for each of HTTP requests (Req1, Req2) - milliseconds.
Min: 0.22 51.06
Avg: 22.78 85.04
Max: 61.18 125.02
--- Network traffic details --- |  |  |  |  |
Total bytes sent : 2604 | Total amount of data sent and received.
Total bytes received : 25320
--- Summary times --- |  |  |  |  |
Virtual Clients statistics: | Virtual clients statistics: Count – amount of HTTP requests sent; Time – virtual client operation time; AR/S – average amount of HTTP requests per second; AT/R – average time of HTTP request processing.
Count Time AR/S AT/R
10 601 16.64 60.10
10 601 16.64 60.10
10 500 20.00 50.00
Total work time: 631 | Script operation time
Total requests made: 30 | Total amount of HTTP requests made by all virtual clients
Total average time per request: 21.0333 |  |  | Average HTTP request processing time (using all requests)
Total average requests per second: 47.5436 |  |  | Average amount of HTTP requests per second
WARNING: approximation used! |  |  |  | The warning is displayed if the overall time of the test script operation is lower than required for precise measurement (and calculation) of the “Total average time per request “ and “Total average requests per second” values. In this case the results will be approximated.
Not enough requests per client for exact timing!
--- HTTP response codes details --- |  | Statistics on HTTP response codes: Code – response code; Count – amount of responses with this code.
Code Count
200 30

Execution report (if scripting is used)

If scripting is used, the textual execution report is simplified. Execution of the script below:

[General]
project=Test Project
Description=test
script=$inline.pl
requests=3
clients=2

[script]
$wapt->AddRequest("test1","http://myserver/");
$wapt->AddRequest("test2","http://myserver/");
for($i=0;$i<$wapt->GetOpt("requests");$i++){
$wapt->Request("test1");
$wapt->Request("test2");
}
[/script]

will result in the following report:

Report generated by Web Roller v3.5 build 957 |
Test started at:Fri Apr 19 16:28:42 2002 |
Test finished at:Fri Apr 19 16:28:43 2002 |
Project name: Test Project | Project name (Projectname setting)
Description: Test | Project desctription (Description parameter)
Server name: myserver | Server tested (Server parameter)
Clients: 2 | Number of virtual clients
Requests: 3 | Number of [TestSeq] runs
Test executed by: Igor | Computer user name
Test executed at: Kort | Computer workstation name
--- Client 0 --- | Statistics on processed requests for virtual client “Client 0”
test1 09.20
test2 12.10
test1 11.60
test2 09.70
test1 12.10
test2 11.60
--- Client 1 --- | Statistics on processed requests for virtual client “Client 1”
test1 09.20
test2 12.10
test1 11.60
test2 09.70
test1 12.10
test2 11.60
--- HTTP response codes details --- | Statistics on HTTP response codes: Code – response code; Count – amount of responses with this code.
Code Count
200 12

Execution Log

If in the [General] section Log=1, all the results of operation are logged to a file. Each virtual client generates a separate log file. The complete log of the Web Roller client operation is written to this file. This includes all requests, responses, redirects and the HTML code returned.

A test script like this:

[General]
clients=1
requests=2
server=kort
log=1

[TestSeq]
Req1
Req2

[Req1]
$uri=/
$method=GET

[Req2]
$uri=/view.pl
$method=GET

will result in the following log file:

----- [TestSeq]: Req1(0) ----- | Separator line: pass #0 of [TestSeq], request Req1
Request: | Header of the HTTP request sent
GET / HTTP/1.1
Host: kort
Connection: close
Response: | Header of the server response.
HTTP/1.1 200 OK
Date: Sun, 21 Apr 2002 16:30:45 GMT
Server: Apache/1.3.12 (Win32)
Connection: close
Transfer-Encoding: chunked
Content-Type: text/html
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> | HTML code of response
<html>
</body>
..
..
</body>
</html>
----- [TestSeq]: Req2 (0) ----- | Separator line: pass #0 of [TestSeq], request Req2
Request: |
GET /view.pl HTTP/1.1 |
Host: kort |
Referrer: http://kort/ |
Connection: close |
Response: |
HTTP/1.1 200 OK |
Date: Sun, 21 Apr 2002 16:30:45 GMT |
Server: Apache/1.3.12 (Win32) |
Connection: close |
Transfer-Encoding: chunked |
Content-Type: text/plain |
<html> |
</body> |
.. |
.. |
</body> |
</html> |
----- [TestSeq]: Req1(1) ----- | Separator line: pass #1 of [TestSeq], request Req1
Request: |
GET / HTTP/1.1 |
Host: kort |
Referrer: http://kort/view.pl |
Connection: close |
Response: |
HTTP/1.1 200 OK |
Date: Sun, 21 Apr 2002 16:30:45 GMT |
Server: Apache/1.3.12 (Win32) |
Connection: close |
Transfer-Encoding: chunked |
Content-Type: text/html |
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.0 Transitional//EN"> |
<html> |
</body> |
.. |
.. |
</body> |
</html> |
----- [TestSeq]: Req2(1) ----- | Separator line: pass #1 of [TestSeq], request Req2
Request: |
GET /view.pl HTTP/1.1 |
Host: kort |
Referrer: http://kort/ |
Connection: close |
Response: |
HTTP/1.1 200 OK |
Date: Sun, 21 Apr 2002 16:30:45 GMT |
Server: Apache/1.3.12 (Win32) |
Connection: close |
Transfer-Encoding: chunked |
Content-Type: text/plain |
<html> |
</body> |
.. |
.. |
</body> |
</html> |

Launching test scripts and projects from Win32 console

The wr.exe executable can be used to launch test scripts and projects in console mode (from the command line). To launch a test project enter:

wr.exe projectname.wpj

To launch a single test script enter:

wr.exe testscriptname.tst [-option=value,., , ]

[Options] can be set if the $gvar function is used within the test script.

Example:

The server parameter in the test script testscript.tst is set as follows:

Server=$gvar(server, myserver)

If we launch the script this way:

wr.exe testscript.tst

the requests will be sent to myserver.

But if we launch the script as:

wr.exe testscript.tst – server=testserver

then the requests will be sent to testserver.

Test script examples

Performance measurements

To ensure correctness and sufficient precision of the measurements, the testing procedure should meet the following requirements:

  • The CPU load of the computer running the tests should not exceed 90% (during test run).
  • If the amount of virtual clients is greater than 1, the start and end points of the measurement period should be located within the time period when ALL virtual clients are active (see the description of delaystart, maxrequests).

    Recommended settings for the [General] section:

[General] |
delaystart=0 | Provides simultaneous start of virtual clients (see description of delaystart)
maxrequests=50000 | The maxrequests value should not exceed the product (clients x requests). Required to eliminate the “final steps” (see the description of maxrequests)
nowait=0 | Do not use the timeout check. Allows to provide significant server load using a single workstation, thanks to saved resources. Consequently, can lead to hang-ups of virtual clients in case of a big amount of stuck requests (which would be normally closed using the timeout check)..
log=0 | Logging is off - to reduce resource consumption of the workstation running the test.
nodetails=1 | Turns off the printing of detailed statistics on separate requests for all virtual clients. Useful if measuring only the value of Requests per second.
summarytimes=1 | Turns on printing of the summary statistics

Example: a test script for measuring the performance of loading the main page of a Web server:

[General]
clients=50
requests=10000
server=myserver
keepalive=1
delaystart=0
maxrequests=50000
nowait=0
log=0
nodetails=1
summarytimes=1

[TestSeq]
Req1

[Req1]
$uri=/

Working with session variables

A typical mistake when automating scripts using session variables is using the static value for the session variable (obtained by recording the sequence of HTTP requests with the Record function) while these session variables are unique for each session; thus an attempt of running such a test script will lead to errors (since other sessions will generate different values of session variables). To provide proper functioning of test scripts in cases like this you need to extract respective session variables from the responses to previous requests (see the description of $UrlParam, $Hidden, $RegExp).

Working with redirects

Usually when test scripts with redirects are recorded, the redirects are not included in Web Roller's proxy log. But if “meta http-equiv” redirects are used, then the situation is possible when the log will include both the initial pages (containing redirects) and the redirects themselves. In this cases the extra redirects should be removed from the log manually.


Introduction

Web Roller is a robust web load testing solution that enables you to easily and accurately test the scalability and performance of applications with Web interface. It has a perfect set of features for testing dynamic and complicated Web and Intranet applications.

Web Roller creates a substantial load on your web and application servers. It is used for performance testing and provides high performance itself. Its Virtual Clients have small memory footprints and make efficient use of CPU resources, therefore ensuring large load with minimal hardware requirements. The high performance of Web Roller guarantees high accuracy of your results.

Web Roller has excellent embedded abilities of creating test data dynamically. Request parameters can be generated in the runtime. Several methods of calculation are provided: from usual (like numeric and alphanumeric random, sequences or lists) - to regular expressions. URI's can be dynamic as well. Test functionality can be extended using the built-in support of several standard programming languages, including JavaScript, VBScript and PerlScript.

Additionally, Web Roller provides you with a wide range of test parameters. Most of global settings can be redefined within individual requests or test sequences.


Scripting

Under the Win32 platform Web Rollerscripting support is implemented via the ActiveScriptingHost, - that is, any scripting language providing the respective interface can be used. Currently JScript, VBScript and PerlScript are supported.

The support of scripting in Web Roller actually means replacing the sequence of requests defined in the original test script with the program defined by the external (or inline) script. All the settings of the current test script remain active (except for sequences - these will be ignored). Whenever scripting is active, Web Roller passes control to the script and does not generate any requests by itself. The Web Roller object with the methods listed below appears in the script's namespace:

Method | Description
int Request ( string requestname); | Attempts to make a request with the specified name. Returns 0, if the request fails (or no request with such name exists), 1 if the request was successful.
string Response (); | Returns the response to the latest request.
int SetParam (string req, string param[, string value]); | Sets the value of the specified parameter of the specified request. If string value is omitted, the parameter is set to the initial value from the request description. Returns 1 upon success, 0 upon failure.
int IsAborting (); | Returns 1 if Ctrl-Break was pressed, 0 if not. If you want the script to correctly terminate on Ctrl-Break, this value should be monitored.
string GetOpt (string opt); | Returns the settings of the global parameters (General section). Additionally returns ClId – the unique number of the current virtual client.
void Logit (string logstr); | Adds a string to the log (if logging is enabled).
int AddRequest(string name, string uri); | Create a new request. Returns 1 upon success, 0 if the request with the specified name already exists.
int SetRequestOpt(string name, string options); | Sets the request parameters. The options string should be in the same format as when passing parameters within a sequence (see in Test scripts).
int AddParam(string reqname, string pname, string pvalue); | Adds a parameter to the specified request.

The name of the script file to be used is set by the scriptfile parameter in the General section. If this setting is present, the scripting is automatically enabled. You can also embed the script text in the body of Web Roller's test script (.TST) file. To do this, you should specify $inline.ext as the value of scriptfile setting. It is important to use the correct ext extension (js, pl or vbs) for inline code, since Web Roller will use it to determine the language of the inline script.

Example (PerlScript):

<$Web Roller->LogIt("Let's try...");
my $res=$Web Roller->AddRequest("test","http://myhost.mydomain.com/scripts/test/params.pl");
$Web Roller->LogIt("AddRequest:$res");
$res=$Web Roller->AddParam("test","param1","paramvalue");
$Web Roller->LogIt("AddParam:$res");
$res=$Web Roller->SetParam("test","param1","xxxyyyzzz");
$Web Roller->LogIt("SetParam:$res");
$res=$Web Roller->Request("test");
$Web Roller->LogIt("Request:$res");
$res=$Web Roller->SetParam("test","param1");
$Web Roller->LogIt("SetParam:$res");
$res=$Web Roller->Request("test");
$Web Roller->LogIt("Request:$res");